Capstone Project - Liberty Mutual Group Fire Peril Loss Cost¶

Author: Alexander Donders

Table of Contents¶

1. Objectives & Intention ¶

In this capstone project, we are going to look at a challenge from the insurance sector that was hosted on Kaggle.

The training set bears several interesting aspects that distinguish this challenge from others. Firstly, the highly imbalanced dataset with over 300 variables makes the use of regression algorithms particularly difficult. Secondly, the meaning of the variables in the dataset is completely unknown. While the first aspect introduces the need for techniques that manage the extreme imbalance in the dataset, the second aspect renders any particular business knowledge in the insurance industry useless as no meaning whatsovever can be obtained from the standardised and anonmymised variables.

However, the quality of the dataset is very high. This allows for less cleaning and preprocessing and more time to explore new techniques to tackle the actual challenge of building a robust model to predict the loss-to-insured value ratio.

Therefore, the question that arises is stated as follows:

How can a valid prediction model be built that manages extremely imbalanced data and builds upon features that are unknown to the machine learning engineer or data scientist?

The following figure summarises the problem as well as the intention and objective:

drawing

#init libraries
import warnings;
warnings.filterwarnings('ignore')

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

import seaborn as sns
from sklearn.externals import joblib
import zipfile
import os

from sklearn.model_selection import train_test_split
from sklearn import linear_model
from sklearn.preprocessing import scale
from sklearn.decomposition import PCA
from sklearn.externals import joblib
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.decomposition import PCA
from sklearn.model_selection import GridSearchCV, StratifiedKFold, ParameterGrid
from sklearn.feature_selection import SelectKBest, f_classif, f_regression
from sklearn.svm import LinearSVC, LinearSVR
from sklearn.linear_model import LogisticRegression, RandomizedLasso, LinearRegression, Ridge, Lasso, ElasticNet
from sklearn.ensemble import GradientBoostingRegressor, ExtraTreesClassifier, RandomForestClassifier, RandomForestRegressor

from xgboost import XGBRegressor
from scipy import stats
from scipy import sparse

%matplotlib inline
#set notebook display options to display the entire printed dataset (if needed)
pd.set_option('display.max_columns', None)  # or 1000
pd.set_option('display.max_rows', None)     # or 1000
pd.set_option('display.max_colwidth', -1)   # or 199

2. Importing the Data ¶

First, we import the data. Note that we know from the Data Description-page on the Kaggle-website, that missing values are denoted using the values "Z" and "NA". Also, during the course of the project, values of type np.nan were encountered that led to unexpected behaviour of certain functions.

#prepare data frames
zf = zipfile.ZipFile('loss-cost.zip') 
df = pd.read_csv(zf.open('train.csv'), na_values=['NA', 'Z', 'nan', np.nan])
df_test = pd.read_csv(zf.open('test.csv'), na_values=['NA', 'Z', 'nan', np.nan])
df_test_ex = pd.read_csv(zf.open('sampleSubmission.csv'))

def get_df_overview(df, sort):
    c = pd.DataFrame({'Count': df.count()})
    dt = pd.DataFrame({'dType': df.dtypes})
    n = pd.DataFrame({'nulls': df.isnull().sum()})
    npct = pd.DataFrame({'nulls pct': df.isnull().sum() / len(df)})
    nn = pd.DataFrame({'not nulls': df.notnull().sum()})

    cols = [df[col].duplicated().sum() for col in df.columns]
    d = pd.DataFrame({'':pd.Series(c.index),'duplicates': cols}).set_index('')
    d.duplicates = d.duplicates - n.nulls + 1
    dpct = pd.DataFrame({'duplicates pct': (d.duplicates) / len(df)})
    
    cols = []
    for col in df.columns:
        if df[col].notnull().sum() > 0:
            cols.append(df[df[col].notnull()].iloc[0,df.columns.get_loc(col)])
        else:
            cols.append("NaN")
    
    f = pd.DataFrame({'': pd.Series(c.index), 'first value': pd.Series(cols)}).set_index('')

    info = c.join(dt).join(n).join(npct).join(nn).join(d).join(dpct).join(f)
    info.sort_values([sort], ascending=[False], inplace=True)
    return info

df.head()

df.shape

(452061, 302)

df_test.shape

(450728, 301)

info = get_df_overview(df, 'nulls pct')
info

Looking at the above summary table, several corrective steps seem appropriate:

Drop variables that contain more than 50% null values (NAs)
Drop unnecessary variables that may not be used as predictors

The quality of the data set appears to be very high. The process of cleaning the data might turn out to be rather straightforward.

According to Kaggle, the variables can be grouped as follows:

id: A unique identifier of the data set

target: The transformed ratio of loss to total insured value

dummy: Nuisance variable used to control the model, but not working as a predictor

var1 – var17: A set of normalized variables representing policy characteristics (note: var11 is the weight used in the weighted gini score calculation)

crimeVar1 – crimeVar9: A set of normalized Crime Rate variables

geodemVar1 – geodemVar37: A set of normalized geodemographic variables

weatherVar1 – weatherVar236: A set of normalized weather station variables

Note that the data is desribed as already being normalised. Hence, in the section Feature Engineering we will not perform any variable (power) transformations.

3. Cleaning the Data ¶

In this section, we will clean the data and prepare it for exploratory data analysis, feature engineering as well as model building. First, we define all the functions necessary for cleaning the data. The functions, however, will be executed at the end of the section feature engineering.

# this function imputes values based on relative frequency of the discrete values (histogram), 
# hence, it can be used for numeric and categorical variables
# the function returns and imputed data frame and an imputation_map object. This objects stores the relative
# frequencies of the missing values per column. Hence, the imputation can be loaded for imputing other data sets
# that bear the same columns. If no imputation map is given, it is freshly calculated from the dataframe at hand.
def impute_missing_values(df, imputation_map=None):
    print('Imputing missing values...')
    def impute_disc_prob(df, c, impute_probs=None):
        df = df.replace({pd.np.nan: None})
        
        if impute_probs==None:
            impute_probs = pd.DataFrame(df[c].value_counts() / df[c].notnull().sum())
            impute_probs.columns = ['p']
        
        n = df[c].isnull().sum()
        r = np.random.choice(impute_probs.index, p=impute_probs.p, size=n)
        df.loc[df[c].isnull(), c] = r
        return df.copy(), impute_probs
    
    df = df.copy()
    df = df.replace({pd.np.nan: None})
    nc = df.isnull().sum() / len(df)
    nc = (df.isnull().sum() / len(df))>0
    cols = nc[nc].index
    
    if imputation_map == None:
        imputation_map = {}
        for c in cols:
            df, impute_probs = impute_disc_prob(df, c)
            imputation_map[c] = impute_probs
    else:
        for c in cols:
            impute_probs = imputation_map.get(c)
            df, impute_probs = impute_disc_prob(df, c, impute_probs)
    
    return df.copy(), imputation_map

def intersection(lst1, lst2): 
    return list(set(lst1) & set(lst2))

def difference(list1, list2):
    return list(set(list1) - set(list2))

def split_column(df, col):
    # according to the data dictionary on Kaggle, this variable is hierarchichal and consists of two parts. 
    # Hence, we split the column in two so that they can be properly encoded.
    df = df.copy()
    df[col] = df[col].astype(str)
    df[col+'0'] = [i[:1] for i in df[col]] 
    df[col+'1'] = [i[1:] for i in df[col]]
    return df.copy()

def get_sorted_values(v):
    v = list(v.unique())
    v = difference(v, [np.nan])
    v.sort()
    return v

def convert_categoricals(df):
    # this function is used to convert variables into either nominal or ordinal variables.
    from pandas.api.types import CategoricalDtype
    df = df.copy()
    print('Converting Variables...')
    for c in intersection(ordinals, df.columns):
        categories = get_sorted_values(df[c])
        dType = CategoricalDtype(categories = categories, ordered = True)
        df[c] = df[c].astype(dType)

    for c in intersection(nominals, df.columns):
        categories = get_sorted_values(df[c])
        dType = CategoricalDtype(categories = categories, ordered = False)
        df[c] = df[c].astype(dType)
        
    return df.copy()

# define the columns that will be dropped from the data set
# all columns that contain more than 50% null-values will be dropped
# note that var4 will be dropped as it will be split in two separat variables
to_drop = list(info[(info['nulls pct'] > 0.5)].index)
to_drop = to_drop + ['id', 'dummy', 'var4']

# we categorise the different variables according the documentation on the Kaggle-website.
# note that the numeric category is derived from the inferred data type from the info-object
ordinals = ['var1', 'var2', 'var3', 'var5', 'var6']
nominals = ['dummy', 'var4', 'var7', 'var8', 'var9', 'var40', 'var41']
numerics = list(info[(info['dType'] == 'float64')].index)

# next, we remove all variables from the categories that we intend to drop 
# furthermore, we make sure that we do not have any nominal or ordinal variables in the 'numericals' bucket
ordinals = difference(ordinals, to_drop)
nominals = difference(nominals, to_drop)
numerics = difference(numerics, to_drop)
numerics = difference(numerics, nominals)
numerics = difference(numerics, ordinals)
numerics = difference(numerics, ['target'])

all_features = ordinals + nominals + numerics

After having prepared the cleansing functions for the data set, we now explore and visualize the variables to get a better understanding of their characteristics. Furthermore, this will help us to determine necessery steps in the section Feature Engineering.

4. Exploratory Data Analysis ¶

As is the case in the previous sections, we first prepare any function that will be needed for actual exploratory analysis. Note, however, that this is not the actual way of how this notebook has eveloved. Rather, this is the result of refactoring and cleaning up the source code for better structure, readability and quality.

Furthermore, due to the sheer amount of variables, this section remains rather compact and brief and its extensive exploratory datanalysis has not been considered feasible:

The anonymised variables do not convey any meaning or semantic that could be interpreted.
The amount of variables is very large.
There are no time series to visialise.

Nevertheless, some visualisations have been added to illustrate what has been found most useful.

def draw_hist(x):
    p = sns.distplot(x, fit=stats.norm,
                 kde_kws={"label": "KDE"},
                 fit_kws={"color": "b", "alpha": .5, "label": "Normal Distribution"}
                )
    plt.legend(loc='best')

def draw_count_plot(c):
    plt.figure(figsize=(17,7))
    p = sns.set()
    p = sns.countplot(x=c, data=df)
    plt.xticks(rotation=45)
    plt.show()

def draw_box_plot(x):
    plt.figure(figsize=(17,7))
    p = sns.set()
    p = sns.boxplot(x=x, y='target', data=df)
    plt.xticks(rotation=45)
    plt.show()

def draw_r_matrix(x, title):
    corr = x.corr()

    mask = np.zeros_like(corr, dtype=np.bool)
    mask[np.triu_indices_from(mask)] = True

    sns.set(style="white")
    f, ax = plt.subplots(figsize=(20, 15))
    cmap = sns.diverging_palette(220, 10, as_cmap=True)

    sns.heatmap(corr, mask=mask, cmap=cmap, vmax=1, center=0, square=True, 
                linewidths=.5, cbar_kws={"shrink": .3}, annot=True, 
                annot_kws={"size": 11}
               )
    plt.title(title, fontsize=24)

The following bar chart gives us an overview of the frequencies of several categorical variables. Following the bar charts, the box plots illustrate the distribution of the target variables among the values of the categorical variables.

for c in intersection(ordinals + nominals,df.columns):
    draw_count_plot(c)

The boxbolots below show which variables are associated with the target variable. Note that there is little variability within the variables. For example. var7exhibits eight values which do not present a dependency of their variability with the target variable. The same is true vor var8.

for c in intersection(nominals,df.columns):
    draw_box_plot(c)

Lastly, we try some correlation heatmaps. As can be seen below, they turn out to be rather useful. In particular, there are a lot of weahter variables that are correlated with each other. The brighter or darker spots of the heatmap show strong positive or negative correlation respectively. However, the sheer amount of these variables renders a manual selection process rather difficult.

Also, within the the weather variables, subgroups ot weather variables can be identified that bear high similarity among each other. They can be found by looking at the brighter or darker cluster that come in squaure shapes. This may justify the use of PCA to reduce the number of dimensions effectively without excessive loss of information.

# here, we put the differend kinds of weather, crime and geo variables intwo their own list
weather = list(df.columns[df.columns.str.contains('weather')])
crime = list(df.columns[df.columns.str.contains('crime')])
geo = list(df.columns[df.columns.str.contains('geo')])
var = list(df.columns[df.columns.str.contains(r'^var')])
var = difference(var+ordinals+nominals, to_drop)

weather_r = df[weather].corr()
plt.figure(figsize = (15,15))
ax = sns.heatmap(weather_r, vmin=-1, vmax=1)

The geographical variables also seem to consist of clusters of variables that are highly positively or negatively correlated amon each other. This may support the use of PCA as a valid compression technique to reduce the number of variables.

geo_r = df[geo].corr()
plt.figure(figsize = (15,15))
ax = sns.heatmap(geo_r, vmin=-1, vmax=1)

Most of the crime variables turn out to be rather highly correlated among each other. The heatmap presented below shows their interrelationships.

crime_r = df[crime].corr()
plt.figure(figsize = (15,15))
ax = sns.heatmap(crime_r, vmin=-1, vmax=1)

Now that we have imported, cleansed and explored the data, we now apply feature engineering techniques to prepare the data for the model building stage.

5. Feature Engineering & Encoding ¶

The following steps are being taken care of in this section:

Inspect target distribution and evaluate if it needs to be transformed
- Censoring large claims
- Downsampling of majority events
Create new features using feature compression techniques (e.g. PCA)
Encode all nominal variables using one-hot-encoding.
Encode ordinal feature to numeric representations

Note that cencoring large claims, downsampling of majority events and the use of PCA has been inspired by the winning team Datarobot as they outlined their approach in their presentation. However, the exact way has not been replicated in this capstone project as their source code is not publicly available.

5.1. Censoring Large Claims and Log-Transformation¶

The following two histograms show that the use of a log transform makes the target variable more normal. Also, the spike at the higher ends of the distribution will be taken out.

draw_hist(df[df['target']>0]['target'])

draw_hist(np.log(df[df['target']>0]['target']+1))

The following function will perform the two operations.

def censor_and_categorize(df):
    df = df.copy()
    upper_bound = df[df.target>0]['target'].quantile(0.80)
    df['target'] = [upper_bound if i > (upper_bound) else i for i in df.target]
    df['LogTarget'] = np.log(df['target']+1)
    df['isClaim'] = [1 if i > 0 else 0 for i in df.target]
    return df.copy()

5.2. Feature Compression: Principal Component Analysis¶

This function applies PCA on a set of variables. The argument features is a list of lists: [one_family_of_variables, another_family_of_variables]. feature_names' which contains the names of the family of features we try to compress to a smaller representation.

For example, if we input a list of three feature families, the three feature families will be compressed individually so that the three families will be preserved but compressed in order to obtain two principal components per family.

Later, this function compresses the weather, geo and crime variables individually to obtain two principal components for the three categories of variables.

def compress_features(df, features, feature_names, pca_objects=None, scaling_objects=None):
    print('Compressing features...')
    df = df.copy()
    comp_feats  = []
    
    if (pca_objects==None): 
        pca_objects = []
        scaling_objects = []
        for i, cols in enumerate(features):
            scaler = StandardScaler().fit(df[cols])
            scaling_objects.append(scaler)
            scaled = scaler.transform(df[cols])
            pca = PCA(n_components=2)
            pca.fit(scaled)
            pca_objects.append(pca)
            pca = pca.transform(scaled)
            components = pd.DataFrame(pca, 
                                      columns=['pca_' + feature_names[i] + '_0', 
                                               'pca_' + feature_names[i] + '_1'
                                              ]
                                     )
            comp_feats.append(components)
    else:
        for i, cols in enumerate(features):
            scaled = scaling_objects[i].transform(df[cols])
            pca = pca_objects[i].fit_transform(scaled)
            components = pd.DataFrame(pca, columns=['pca_' + feature_names[i] + '_0', 'pca_' + feature_names[i] + '_1'])
            comp_feats.append(components)
            
    feature_components = pd.concat(comp_feats, axis=1)
    feature_components.reset_index(drop=True, inplace=True)
    return feature_components.copy(), pca_objects, scaling_objects

5.3. Downsampling of majority events¶

This function takes care of resampling the dataset. More precisely, only the subset of data will be resampled that does not exhibit a target-value >0. Since most target values are zero, only these rows will be resampled downwards (downsampled) to balance out the minority class (target > 0) with the majority class (target == 0).

def downsample(df, maj_min_ratio=1):
    print('Resampling...')
    from sklearn.utils import resample
    df = df.copy()

    # Separate majority and minority classes
    df_maj = df[df.LogTarget == 0]
    df_min = df[df.LogTarget > 0]

    # Downsample majority class
    df_maj_ds = resample(df_maj, 
                         replace = False,                         # sample without replacement
                         n_samples=len(df_min) * maj_min_ratio,   # to match minority class
                         random_state=0)                          # reproducible results

    # Combine minority class with downsampled majority class
    df = pd.concat([df_maj_ds, df_min], axis=0, ignore_index=True)
    return df.copy()

5.4. Perform Preprocessing¶

Finally, we actually apply all the defined functions from above and preprocess the training, validation and test set of the project.

def encode_features(df):
    print('Encoding features...')
    df = df.copy()
    info = get_df_overview(df, 'nulls pct')
    for c in info[info.dType=='category'].index:
        df[c] = df[c].cat.codes
    return df.copy()

def preprocess(df, features, mode, imputation_probabilities=None, pca_objects=None, scaling_objects=None):

    df = df.copy()
    df = split_column(df, 'var4')
    df = df.drop(intersection(features['To Drop'] + ['target'], df.columns), axis = 1)
    df, imputation_probabilities = impute_missing_values(df)
    cf, pca_objects, scaling_objects = compress_features(df, features['PCA']['PCA Features'], features['PCA']['PCA Feature Names'], pca_objects, scaling_objects)

    # variable separation
    if (mode=='Train'):
        targets = df[['LogTarget', 'isClaim']]
        targets.reset_index(drop=True, inplace=True)
        
        variables = df[features['Normal Variables']]
        variables.reset_index(drop=True, inplace=True)
        
        cf.reset_index(drop=True, inplace=True)
        df_long_comp = pd.concat([targets, variables, cf], axis=1)
        
        # sampling
        df_short = downsample(df, 10)
        df_short_comp = downsample(df_long_comp, 10)
        
        # conversions
        df = convert_categoricals(df)
        df = encode_features(df)

        df_long_comp = convert_categoricals(df_long_comp)
        df_long_comp = encode_features(df_long_comp)

        df_short = convert_categoricals(df_short)
        df_short = encode_features(df_short)

        df_short_comp = convert_categoricals(df_short_comp)
        df_short_comp = encode_features(df_short_comp)
        
        return_dict = {
            'long_wide': df,
            'short_wide': df_short,
            'long_comp': df_long_comp,
            'short_comp': df_short_comp,
        }
        
    elif (mode=='Validation'):
        targets = df[['LogTarget', 'isClaim']]
        targets.reset_index(drop=True, inplace=True)
        
        variables = df[features['Normal Variables']]
        variables.reset_index(drop=True, inplace=True)
        
        cf.reset_index(drop=True, inplace=True)
        df_long_comp = pd.concat([targets, variables, cf], axis=1)  
        
        # conversions
        df = convert_categoricals(df)
        df = encode_features(df)

        df_long_comp = convert_categoricals(df_long_comp)
        df_long_comp = encode_features(df_long_comp)

        return_dict = {
            'long_wide': df,
            'long_comp': df_long_comp
        }    

    elif (mode=='Test'):
        variables = df[features['Normal Variables']]
        variables.reset_index(drop=True, inplace=True)
        
        cf.reset_index(drop=True, inplace=True)
        df_long_comp = pd.concat([variables, cf], axis=1)  
        
        # conversions
        df = convert_categoricals(df)
        df = encode_features(df)

        df_long_comp = convert_categoricals(df_long_comp)
        df_long_comp = encode_features(df_long_comp)

        return_dict = {
            'long_wide': df,
            'long_comp': df_long_comp
        }     
    
    return return_dict, imputation_probabilities, pca_objects, scaling_objects

# here, we put the differend kinds of weather, crime and geo variables intwo their own list
# subsequently, the weather, criime and geo variables are compressed individually two obtain
# two principal components for the three categories of variables each.
weather = list(df.columns[df.columns.str.contains('weather')])
crime = list(df.columns[df.columns.str.contains('crime')])
geo = list(df.columns[df.columns.str.contains('geo')])
var = list(df.columns[df.columns.str.contains(r'^var')])
var = difference(var+ordinals+nominals, to_drop)

features = {
    'PCA': {
        'PCA Features': [crime, geo, weather],
        'PCA Feature Names': ['crime', 'geo', 'weather']
    },
    'To Drop': to_drop,
    'Normal Variables': var,
    'Weather Variables': weather,
    'Crime Variables': crime,
    'Geographic Variables': geo
}

df_train = censor_and_categorize(df)
df_train, df_valid = train_test_split(df_train, test_size=0.3, stratify=df_train['isClaim'], random_state=0)

df_train_dict, imputation_probabilities, pca_objects, scaling_objects = preprocess(df_train, features, 'Train')

Imputing missing values...
Compressing features...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:12: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  if sys.path[0] == '':
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:12: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  if sys.path[0] == '':
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:12: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
  if sys.path[0] == '':

Resampling...
Resampling...
Converting Variables...
Encoding features...
Converting Variables...
Encoding features...
Converting Variables...
Encoding features...
Converting Variables...
Encoding features...

df_valid_dict, _, _, _ = preprocess(df_valid, features, 'Validation', imputation_probabilities, pca_objects, scaling_objects)

Imputing missing values...
Compressing features...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.

Converting Variables...
Encoding features...
Converting Variables...
Encoding features...

df_test_dict, _, _, _ = preprocess(df_test, features, 'Test', imputation_probabilities, pca_objects, scaling_objects)

Imputing missing values...
Compressing features...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\ipykernel_launcher.py:25: DataConversionWarning: Data with input dtype object were all converted to float64 by StandardScaler.

Converting Variables...
Encoding features...
Converting Variables...
Encoding features...

joblib.dump(df_train_dict, 'Data\df_train_dict.joblib')
joblib.dump(df_valid_dict, 'Data\df_valid_dict.joblib')
joblib.dump(df_test_dict, 'Data\df_test_dict.joblib')

['Data\\df_test_dict.joblib']

6. Model Evaluation Metric: Normalised Weighted Gini Coefficient ¶

In order to compare model performance with the official Kaggle leaderboard, we calculate the normalized, weighted Gini coefficient.

According to the Kaggle website, the normalized, weighted Gini coefficient is calculated as follows:

def weighted_gini(act,pred,weight): 
    df = pd.DataFrame({"act":act,"pred":pred,"weight":weight}) 
    df = df.sort_values('pred',ascending=False) 
    df["random"] = (df.weight / df.weight.sum()).cumsum() 
    total_pos = (df.act * df.weight).sum() 
    df["cumposfound"] = (df.act * df.weight).cumsum() 
    df["lorentz"] = df.cumposfound / total_pos 
    n = df.shape[0] 

    gini = sum(df.lorentz[1:].values * (df.random[:-1])) - sum(df.lorentz[:-1].values * (df.random[1:])) 
    return gini

def normalized_weighted_gini(act,pred,weight):
    return weighted_gini(act,pred,weight) / weighted_gini(act,act,weight)

7. Model Building ¶

In this section the predictive models are being built.

The first section oulines the process of how the predictive models are going to be built. The second section looks at how the individual lerners are trained. The third section looks at how the individual lerners are aggregated (stacked) to an ensemble of lerners. Lastly, all the individual models and their aggregations will be evaluated before the final prediction.

7.1. Model Building Process ¶

For building the model in this project, we distinguish between the public and private data set. While we know the regression target variable in the public data set, the target is completely unknown to us in the private data set. The private data set is used for the final predction and submission after the model has been build, evaluated, tuned and selected for production.

To build the model we take the public data set and split it into a training set, a validation set and a test set.

Training Set: Used to train the individual learners.

Validation: The individual lerners make their prediction on the hold-out set and an ensemble lerner (meta lerner) will learn the weights put on every individual learner.

Test: is not used to learn any relationship. Rather, this will be used to assess performance of the individual and ensemble models before it (or they) is used for the final (private) predictions.

drawing

The subsection below will outline that we will use six different variations of the original training set to train the models. Due to this, a custom train_test()-function is built to separate the individual training and test splits. The function returns a dictionary to properly handle the complexity of having multiple different splits.

def train_test(df, regression_y, classification_y, split_ratio):
    df = df.copy()
    X = df.drop([regression_y, classification_y], axis=1)
    X = pd.get_dummies(X, columns=intersection(nominals,X.columns), prefix=intersection(nominals,X.columns))
   
    y = df[regression_y]
    X_tr, X_te, y_reg_tr, y_reg_te = train_test_split(X, y, train_size=split_ratio, random_state=0, stratify=df[classification_y])

    y = df[classification_y]
    X_clf_tr, X_clf_te, y_clf_tr, y_clf_te = train_test_split(X, y, train_size=split_ratio, random_state=0, stratify=y)
    
    return {
                'X_tr': X_tr.copy(), 
                'X_te': X_te.copy(), 
                'y_reg_tr': y_reg_tr.copy(), 
                'y_reg_te': y_reg_te.copy(), 
                'y_clf_tr': y_clf_tr.copy(), 
                'y_clf_te': y_clf_te.copy()
           }

The function below performs automatic feature selection using six voters (regressors and classifiers) that vote on the dataset and assess wheather or not a particular variable is usefull:

uv: is a statistical univariate test based on ANOVA. 01, 005 are the significance levels (p-values) upon which a variable is considered different from zero
Furthermore, several other linear classifiers with L1 penalty are used.

Finally, a feature is considered good if it receives at least four votes.

def select_best_features(X_tr, y_reg_tr, y_clf_tr, save_features, refresh=True):
    if refresh:
        
        features = pd.DataFrame({'Column': X_tr.columns})
        
        X_tr = X_tr.copy().values
        y_reg_tr = y_reg_tr.copy().values
        y_clf_tr = y_clf_tr.copy().values
        X_sp = sparse.coo_matrix(X_tr)
        
        selector_clf = SelectKBest(f_classif, k = 'all').fit(X_tr, y_clf_tr)

        features['uv_clf_alpha_01'] = (selector_clf.pvalues_ < 0.1)
        features['uv_clf_alpha_005'] = (selector_clf.pvalues_ < 0.05)
        features['uv_clf_alpha_001'] = (selector_clf.pvalues_ < 0.01)
        features['Randomized Lasso'] = RandomizedLasso(random_state = 0).fit(X_tr, y_reg_tr).get_support()
        features['Linear SVC'] = (LinearSVC(C=0.1, penalty = "l1", dual = False, random_state = 0, class_weight='balanced').fit(X_tr, y_clf_tr).coef_>0).reshape(-1)
        features['Logistic Regregssion'] = (LogisticRegression(C=0.01, penalty = 'l1', random_state = 0, class_weight='balanced').fit(X_tr, y_clf_tr).coef_>0).reshape(-1)
        features['Votes'] = features.sum(axis=1)
        features['Good Feature'] = features.Votes >= 4
    else:
        features = pd.read_csv(save_features)
        
    good_features = list(features[features['Good Feature']==True].Column)
    features.to_csv(save_features, index=False)
    
    return good_features, features

Now we prepare the six different datasets. The six different training sets will be explained in further detail in the next paragraph.

refresh = False
training_dict = {} # create a dictionary of different training sets
training_dict['long_wide'] = train_test(df_train_dict['long_wide'], 'LogTarget', 'isClaim', 0.8) 

good_features_long, features_long = select_best_features(training_dict['long_wide']['X_tr'], 
                                               training_dict['long_wide']['y_reg_tr'], 
                                               training_dict['long_wide']['y_clf_tr'],
                                               'Features\Features_long.csv', refresh
                                              )

training_dict['short_wide'] = train_test(df_train_dict['short_wide'], 'LogTarget', 'isClaim', 0.8) 
training_dict['long_comp'] = train_test(df_train_dict['long_comp'], 'LogTarget', 'isClaim', 0.8) 
training_dict['short_comp'] = train_test(df_train_dict['short_comp'], 'LogTarget', 'isClaim', 0.8)

training_dict['long_selected'] = training_dict['long_wide'].copy()
training_dict['long_selected']['X_tr'] = training_dict['long_selected']['X_tr'][good_features_long].copy()

good_features_short, features_short = select_best_features(training_dict['short_wide']['X_tr'], 
                                               training_dict['short_wide']['y_reg_tr'], 
                                               training_dict['short_wide']['y_clf_tr'],
                                               'Features\Features_short.csv', refresh
                                              )

training_dict['short_selected'] = training_dict['short_wide'].copy()
training_dict['short_selected']['X_tr'] = training_dict['short_selected']['X_tr'][good_features_short].copy()

joblib.dump(training_dict, 'Data\\training_data.joblib')

['Data\\training_data.joblib']

7.2. Regression & Classification Ensemble ¶

After having outlined the process of how we are going to build our ensemble. The figure below illustrates the ensemble building strategy.

First, we will train several regression and classification algorithms on the data set. Every single algorithm is trained with different versions of the training dataset:

Original Dataset: Original training set that has been cleansed and one-hot encoded (data set long_wide)

Compressed Dataset: Original dataset where the geo, weather and crime variables have been compressed using PCA (data set long_comp)

Resampled Dataset: Downsampled versions where the majority events are downsampled to a majority-to-minority ratio of 10 (data set short_wide).

Resampled & Compressed: Downsampled data set that has been compressed using PCA for the geo, weather and crime variables (data set short_comp).

Selected Features: Original dataset, but features are reduced using automatic feature selection (function select_best_features() (data set long_selected).

Selected & Resampled: Downsampled dataset dataset that has been reduced in width as well using automatic feature selection (data set short_selected).

In the following figure, the matrices below the regression and classification algorithms indicate in green if the dataset is used to train this particular model. Grey means that the algorithm is not trained on this specific data set. For classification, note that we only use the long (not resampled) versions of the training data as we will only use classifiers that implement a class_weight-parameter. The class_weight parameter puts inversely proportional weights on the errors of the majority and minitory class to balance out the classification accuracy. Therefore, we will only use Logistic Regression, Extra Trees, Random Forest and Linear SVC.

It should be pointed out that every learner is trained using 5-Fold Cross validation and Grid Search.

Once the individual learners have been trained, all the learners will make predictions on the validation (hold-out) set. After having made all predictions (46 predictions in total), we will fit a all these 46 regression and classification outputs to a new meta learner. This meta learner will assign a coefficient to each individual prediction made by the individual learners. These coefficients (or weights) are illustrated by the brightness of the orange tiles. Good individual learners will become a higher weight (darker), while weak learners will be a assigned a lower weight (and, hence, will be brighter).

drawing

class Model:
    # this custom class implements a function 'fit' that fits a model to a list of data sets that is specified using the
    # parameter 'datasets'. Also, the constructor receives the regression or classification 'model' to fit the data to, 
    # a grid search dictionary, and some descriptive attributes such as model_type, model_name.  
    # the parameters 'cv' and 'verbose' are passed to GridSearchCV().
    # The fitted models are stored in the dictionary 'fits' that contains the fitted models per dataset within 
    # a `Model`-object.
    def __init__(self, model_type, model_name, model, grid, datasets, cv, verbose):
        self.model_type = model_type
        self.model_name = model_name
        self.model = model
        self.grid = grid
        self.datasets = datasets
        self.cv = cv
        self.verbose = verbose
        self.fits = {}
        
    def fit(self):
        for ds_key, dataset in self.datasets.items():
            pipe = Pipeline([
                ('scaler', StandardScaler()),
                ('model', self.model)
            ])

            gs = GridSearchCV(pipe, self.grid, cv=self.cv, verbose=self.verbose)
            X = dataset['X_tr']
            
            if self.model_type=='Regression':
                y = dataset['y_reg_tr']
            else:
                y = dataset['y_clf_tr']

            print('Fitting', self.model_name, 'on data set', ds_key, 'with shape', X.shape, '...')
            self.fits[ds_key] = gs
            self.fits[ds_key].fit(X, y)

# in this cell, we prepare the arguments for instanciating objects of the class 'Model'. 
# first, we prepare a list of different data sets for the objects of the class 'Model'.
# Then, we define the grid search dictionaries.

all_sets = ['long_wide', 'short_wide', 'long_comp', 'short_comp', 'long_selected', 'short_selected']
small_sets = ['long_comp', 'short_comp', 'long_selected', 'short_selected']
long_sets = ['long_wide', 'long_comp', 'long_selected']

all_data = {key:training_dict[key] for key in all_sets}
small_data = {key:training_dict[key] for key in small_sets}
long_data = {key:training_dict[key] for key in long_sets}

# grid search dictionaries
ols_grid = {'model__fit_intercept': [True]}
lr_grid = {'model__alpha': np.logspace(-6,6,10)}
svr_grid = {'model__C': [1,2,4]}

rf_grid = {
    'model__n_estimators': [500],
    'model__max_depth': [5],
    'model__min_samples_leaf': [5]
}

gbr_grid = {
    'model__learning_rate': [0.001],
    'model__n_estimators': [500],
    'model__max_depth': [5],
    'model__min_samples_leaf': [5]
}

xgbr_grid = {
    'model__min_child_weight': [0.9], 
    'model__gamma': [0.4],  
    'model__subsample': [1.0],
    'model__colsample_bytree': [0.9], 
    'model__max_depth': [4]
}

log_reg_grid = {
    'model__penalty': ['l1'], 
    'model__max_iter': [1000], 
    'model__class_weight': ['balanced']
}

xt_grid = {
    'model__n_estimators': [300], 
    'model__class_weight': ['balanced']
}

rf_clf_grid = {
    'model__n_estimators': [300], 
    'model__class_weight': ['balanced']
}

svc_grid = {
    'model__class_weight': ['balanced']
}

# Now, we define a dictionary 'ensemble' which we use to instaciate our objects of the class 'Model'
# and we pass the constructor arguments for proper initialisation.

ensemble = {
    'LinearRegression': Model('Regression', 'LinearRegression', LinearRegression(), ols_grid, all_data, 5, 0),
    'Lasso': Model('Regression', 'Lasso', Lasso(random_state=0), lr_grid, all_data, 5, 0),
    'Ridge': Model('Regression', 'Ridge', Ridge(random_state=0), lr_grid, all_data, 5, 0),
    'RandomForestRegression': Model('Regression', 'RandomForestRegression', RandomForestRegressor(random_state=0), rf_grid, small_data, 5, 0),
    'GradientBoostingRegression': Model('Regression', 'GradientBoostingRegression', GradientBoostingRegressor(random_state=0), gbr_grid, small_data, 5, 0),
    'XGBoostRegression': Model('Regression', 'XGBRegression', XGBRegressor(random_state=0), xgbr_grid, small_data, 5, 0),
    'LinearSVR': Model('Regression', 'LinearSVR', LinearSVR(random_state=0), svr_grid, small_data, 5, 0),
    
    'LogisticRegression': Model('Classification', 'LogisticRegression', LogisticRegression(random_state=0), log_reg_grid, long_data, 5, 0),
    'ExtraTreesClassifier': Model('Classification', 'ExtraTreesClassifier', ExtraTreesClassifier(random_state=0), xt_grid, long_data, 5, 0),
    'RandomForestClassifier': Model('Classification', 'RandomForestClassifier', RandomForestClassifier(random_state=0), rf_clf_grid, long_data, 5, 0),
    'SVC': Model('Classification', 'LinearSVC', LinearSVC(random_state=0), svc_grid, long_data, 5, 0)
}

# Now that our models have been instanciated, we call the method 'fit()' on every object of the class 'Model'
# in the 'ensemble' dictionary.
# after every iteration, we delete the datasets in the 'Model'-objects
# to spare storage space and save the model to a file.
for key, model in ensemble.items():
    model.fit()
    model.datasets = None
    joblib.dump(model, 'Models\\' + str(model.model_type) + '_' + str(model.model_name) + '.joblib')

Fitting LinearRegression on data set long_wide with shape (253153, 331) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting LinearRegression on data set short_wide with shape (7321, 331) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\model_selection\_search.py:841: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.
  DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting LinearRegression on data set long_comp with shape (253153, 55) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting LinearRegression on data set short_comp with shape (7321, 55) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting LinearRegression on data set long_selected with shape (253153, 66) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

# code to recover models, if needed
# ensemble = {
#     'LinearRegression': joblib.load('Models\\Regression_LinearRegression.joblib'),
#     'Lasso': joblib.load('Models\\Regression_Lasso.joblib'),
#     'Ridge': joblib.load('Models\\Regression_Ridge.joblib'),
#     'RandomForestRegression': joblib.load('Models\\Regression_RandomForestRegression.joblib'),
#     'GradientBoostingRegression': joblib.load('Models\\Regression_GradientBoostingRegression.joblib'),
#     'XGBoostRegression': joblib.load('Models\\Regression_XGBRegression.joblib'),
#     'LinearSVR': joblib.load('Models\\Regression_LinearSVR.joblib'),
    
#     'LogisticRegression': joblib.load('Models\\Classification_LogisticRegression.joblib'),
#     'ExtraTreesClassifier': joblib.load('Models\\Classification_ExtraTreesClassifier.joblib'),
#     'RandomForestClassifier': joblib.load('Models\\Classification_RandomForestClassifier.joblib'),
#     'SVC': joblib.load('Models\\Classification_LinearSVC.joblib')
# }

# training_dict = joblib.load('Data\\training_data.joblib')
# df_valid_dict = joblib.load('Data\\df_valid_dict.joblib')

7.3. Ensembling: Ensemble Stacking & Blending ¶

After having trained all models successfully, we now make predictions on the validation (hold-out) dataset using all fitted models in all variations. By doing that, we obtain 56 different predictions per observations. Note that for the classification models, we compute both the classification probability as well as the classification itself.

The following figure shows in orange the hold-out set with its features (301 features in the long and wide version and less in the compressed and reduced versions). To the right of the hold-out set, we see the predictions made row by row for every model. Every column to the right of the hold-out set is a different prediction from a different algorithm.

drawing

def predict(df_long_wide, df_long_comp, ensemble):
    # this function receives a wide and a compressed version of a test or validation dataset along with a dictionary of 
    # individually trained models.
    # the function returns a dataframe with a column per model giving their prediction (classification and regression).
    
    df_long_wide = df_long_wide.copy()
    df_long_comp = df_long_comp.copy()
    df_ensemble = pd.DataFrame()
    
    for key, model in ensemble.items():
        for dataset, fit in model.fits.items():

            if 'comp' in dataset:
                X = df_long_comp
            else:
                X = df_long_wide

            #X = pd.get_dummies(X, columns=intersection(nominals,X.columns), prefix=intersection(nominals,X.columns))
            X = X.reindex(columns = training_dict[dataset]['X_tr'].columns, fill_value=0)

            # predict and prepare df_ensemble for final regression
            col = model.model_name + '_' + dataset

            if model.model_type=='Regression':
                df_ensemble[col + '_reg'] = fit.predict(X)
                #df_ensemble[np.abs(df_ensemble[col + '_reg']) < 0.00002] = 0
            else:
                df_ensemble[col + '_clf'] = fit.predict(X)
                try:
                    df_ensemble[col + '_prob'] = fit.predict_proba(X)[:,1]
                except:
                    pass # --> ignore predictors that do not implement the predict_proba()-method.
                
    reg_cols = [col for col in df_ensemble.columns if '_reg' in col]
    prob_cols = [col for col in df_ensemble.columns if '_prob' in col]
    clf_cols = [col for col in df_ensemble.columns if '_clf' in col]
    
    # besides now compute the mean across all regressors, the mean probability and mean classification
    # for all classifiers. Then we take the mean regressor value and weight it by the mean probability prediction.
    # lastly, we weight the regression mean by the majority vote, which is either zero or one
    # this way, we get three different kind of ensemble predictions
    df_ensemble['Regressor Mean'] = df_ensemble[reg_cols].mean(axis=1)
    df_ensemble['Probability Mean'] = df_ensemble[prob_cols].mean(axis=1)  
    df_ensemble['Majority Vote'] = round(df_ensemble[clf_cols].mean(axis=1),0)
    df_ensemble['Mean Probability Weighted Regression Mean'] = df_ensemble['Regressor Mean'] * df_ensemble['Probability Mean']
    df_ensemble['Majority Vote Weighted Regression Mean'] = df_ensemble['Regressor Mean'] * df_ensemble['Majority Vote']

    return df_ensemble.copy()

# df_valid_dict['long_comp']['LogTarget'] = df_valid_dict['long_wide'].reset_index()['LogTarget'].copy()
# df_valid_dict['long_comp']['isClaim'] = df_valid_dict['long_wide'].reset_index()['isClaim'].astype(int).copy()

# we split the validation data set into a training and testing set in order to properly validate our results.
df_valid_long_wide = train_test(df_valid_dict['long_wide'], 'LogTarget', 'isClaim', 0.8)
df_valid_long_comp = train_test(df_valid_dict['long_comp'], 'LogTarget', 'isClaim', 0.8)

print('Hold-out Training Size', df_valid_long_wide['X_tr'].shape)
print('Hold-out Validation Size', df_valid_long_wide['X_te'].shape)

Hold-out Training Size (108495, 331)
Hold-out Validation Size (27124, 331)

y_train = pd.DataFrame({'isClaim': df_valid_long_wide['y_reg_tr'], 'LogTarget': df_valid_long_wide['y_clf_tr']})     
ensemble_predictions = predict(df_valid_long_wide['X_tr'], df_valid_long_comp['X_tr'], ensemble)

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)

df_ensemble_train = pd.concat([ensemble_predictions.reset_index(), y_train.reset_index()], axis=1).drop('index', axis=1)

df_ensemble_train.head()

The above dataframe df_ensemble holds predictions that were made on the validation set. In the next section, these individual prediction will be used to build a meta regressor that use the individual regressor predictions as its features.

7.4. Ensembling: Training a Meta Learner ¶

In this section we train several meta learners. The meta learner is a regression that uses the individual predictions of the target variable as a feature. The idea behind the meta learner is to capture the influence of each of these meta learners and adjust their prediction to more generaly fit the model.

drawing

We also perform an automatic feature selection for an automatic evaluation of which learners may be particularly useful. The same function is used as above where we created the data set long_selected and short_selected.

good_predictors, predictors = select_best_features(df_ensemble_train.drop(['LogTarget', 'isClaim'], axis=1), 
                                               df_ensemble_train.LogTarget, 
                                               df_ensemble_train.isClaim.astype(int),
                                               'Models\\Predictors.csv', True
                                              )

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\feature_selection\univariate_selection.py:114: UserWarning: Features [48 53 54] are constant.
  UserWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:58: DeprecationWarning: Class RandomizedLasso is deprecated; The class RandomizedLasso is deprecated in 0.19 and will be removed in 0.21.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:58: DeprecationWarning: Class Memory is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:58: DeprecationWarning: Class Parallel is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\utils\deprecation.py:77: DeprecationWarning: Function delayed is deprecated; deprecated in version 0.20.1 to be removed in version 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib.
  warnings.warn(msg, category=DeprecationWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\svm\base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
  "the number of iterations.", ConvergenceWarning)

# since we use the same interface as above to train the new meta learners, we have to put our newly built meta training sets
# into a dictionary so that the objects of the class `Model` can read them.
# first, we put a set of all individual lerners. The second training set is a subset consisting only of good predictors
# as evaluated by the automatic (meta) features selector function select_best_features().
meta_train_dict= {}
temp = {}
temp['X_tr'] = df_ensemble_train.drop(['LogTarget', 'isClaim'], axis=1)
temp['y_reg_tr'] = df_ensemble_train.LogTarget
meta_train_dict['ensemble'] = temp

temp = {}
temp['X_tr'] = df_ensemble_train[good_predictors]
temp['y_reg_tr'] = df_ensemble_train.LogTarget
meta_train_dict['selected_ensemble'] = temp

# prepare grid as expected for the model constructor
lr_grid = {'model__alpha': np.logspace(-6,6,10)}

en_grid = {'model__max_iter': [1, 5, 10],
              'model__alpha': np.logspace(-6,6,10),
              'model__l1_ratio': np.arange(0.0, 1.0, 0.1)
          }

# define the set of meta predictors we would like to evaluate
meta_predictors = {
    'LinearRegression': Model('Regression', 'LinearRegression', LinearRegression(), ols_grid, meta_train_dict, 5, 0),
    'Lasso': Model('Regression', 'Lasso', Lasso(random_state=0), lr_grid, meta_train_dict, 5, 0),
    'Ridge': Model('Regression', 'Ridge', Ridge(random_state=0), lr_grid, meta_train_dict, 5, 0),
    'ElasticNet': Model('Regression', 'ElasticNet', ElasticNet(random_state=0), en_grid, meta_train_dict, 5, 0)
}

# train the meta predictors
for key, model in meta_predictors.items():
    model.fit()
    model.datasets = None
    joblib.dump(model, 'Models\\' + 'MetaPredictor_' + str(model.model_name) + '.joblib')

Fitting LinearRegression on data set ensemble with shape (108495, 60) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting LinearRegression on data set selected_ensemble with shape (108495, 33) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting Lasso on data set ensemble with shape (108495, 60) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting Lasso on data set selected_ensemble with shape (108495, 33) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\linear_model\coordinate_descent.py:492: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Fitting data with very small alpha may cause precision problems.
  ConvergenceWarning)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

Fitting Ridge on data set ensemble with shape (108495, 60) ...

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:511: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\preprocessing\data.py:625: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\base.py:465: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)

7.4. Ensembling: Validating the Meta Learner ¶

Having trained all the individual models, the (weighted) average ensemble predictors as well as our meta learners, this subsection presents the performance of all models how they perform on the held-out test set, which has never been used before.

# this function receives a dataframe with all prediction variations as its columns plus the true target value for evaluation
# since we need to compare our model based on the normalized weighted gini coefficient, the weight used for comparison is 
# var11.

# the function returns a dataframe showing 
    # the normalized weighted gini coefficient 
    # model type (individual regressor or classifier or ensemble)
    # weight (weight is calculated based on the normalized weighted gini coefficient)
        # per model type (Regressor, Classifier, Ensemble), all weights sum up to 1. The weights are used to build a 
        # version of an ensemble, which will be the gini weighted regression ensemble, where we take 
        # the weighted average of all predictions and multiply
        # this value by the gini weighted probability of being a loss from all classification probabilities.
def get_scores(df, weights):
    df = df.copy()
    weights = weights.copy()
    winners = []
    cols = []

    for col in list(df.columns):
        gini = normalized_weighted_gini(
            np.exp(np.float64(df.LogTarget))-1, 
            np.exp(np.float64(df[col]))-1, 
            weights
        )

        cols.append(col)
        winners.append(gini)

    df_win = pd.DataFrame({'gini': winners}, index=cols).drop(['LogTarget', 'isClaim'])
    df_win = df_win.sort_values('gini', ascending=False)
    
    df_win['model type'] = ['Regressor' if ('_reg' in i) else 'Classifier' if ('_prob' in i) else 'Ensemble' for i in df_win.index]
    df_win['weight'] = df_win.groupby('model type').gini.apply(normalizer)
    df_win = df_win.rename(index=str, columns={'index': 'column'})
    
    return df_win.copy()

def normalizer(x):
    # this function is used to convert the normalised weighted gini scores into weights that sum to one
    norm = 1 / sum(x)
    return x * norm

def gini_weighted_ensemble(df, df_scores):
    # this function weights the individual predictions as obtained from the individual learners and weights these
    # predictions by the gini score obtained by the model.
    # the weightings are done separately for regression and classification models
    # finally, the gini-weighted regression prediction is weighted by the gini-weighted classification probability
    
    df = df.copy()
    df_scores = df_scores.copy()
    reg_cols = [col for col in df.columns if '_reg' in col]
    prob_cols = [col for col in df.columns if '_prob' in col]

    regressors = df[reg_cols].reindex(sorted(df[reg_cols].columns), axis=1)
    regressor_weights = df_scores[df_scores['model type']=='Regressor'].sort_index(axis=1).T.loc['weight']
    #regressor_weights[regressor_weights < 0] = 0
    weighted_avg_regressor = (regressor_weights.values * regressors.values).sum(axis=1)

    classifiers = df[prob_cols].reindex(sorted(df[prob_cols].columns), axis=1)
    classifier_weights = df_scores[df_scores['model type']=='Classifier'].sort_index(axis=1).T.loc['weight']
    #classifier_weights[classifier_weights < 0] = 0
    weighted_avg_classifier = (classifier_weights.values * classifiers.values).sum(axis=1)
    
    return (weighted_avg_regressor * weighted_avg_classifier).copy()

# first, we perform the individual predictions from the individual learners
valid_predictions = predict(df_valid_long_wide['X_te'], df_valid_long_comp['X_te'], ensemble)

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype uint8, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)

y_valid = pd.DataFrame({'isClaim': df_valid_long_wide['y_reg_te'], 'LogTarget': df_valid_long_wide['y_clf_te']}) 
df_ensemble_valid = pd.concat([valid_predictions.reset_index(), y_valid.reset_index()], axis=1).drop('index', axis=1)

# now, we add the meta learners, which add their predictions to the data frame
X_meta = df_ensemble_valid.drop(['LogTarget', 'isClaim'], axis=1)
y_meta = df_ensemble_valid.LogTarget

for key, model in meta_predictors.items():
    for dataset, fit in model.fits.items():
        X_meta = X_meta.reindex(columns = meta_train_dict[dataset]['X_tr'].columns, fill_value=0)
        if dataset=='selected_ensemble':
            df_ensemble_valid['MetaRegressor' + '_selected_' + model.model_name] = fit.predict(X_meta)
        else:
            df_ensemble_valid['MetaRegressor' + '_' + model.model_name] = fit.predict(X_meta)

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)

# lets compare the scores
df_scores = get_scores(df_ensemble_valid, df_valid_long_wide['X_te'].var11)
df_scores.plot.bar(y='gini', figsize=(20,10), title='Model Ranking by Normalised Weighted Gini Score')

<matplotlib.axes._subplots.AxesSubplot at 0x2495d741518>

# now we add the gini weighted regression ensemble
df_ensemble_valid['Gini Weighted Regression Ensemble'] = gini_weighted_ensemble(df_ensemble_valid, df_scores)
df_scores = get_scores(df_ensemble_valid, df_valid_long_wide['X_te'].var11)
df_scores.plot.bar(y='gini', figsize=(20,10), title='Model Ranking by Normalised Weighted Gini Score')

<matplotlib.axes._subplots.AxesSubplot at 0x2476a3dfa90>

# lastly, we compute the gini weighted regression ensemble but only based on the meta features that have been 
# automatically identified and selected as being 'good'.
df_ensemble_valid['Gini Weighted Regression Ensemble (Selected)'] = gini_weighted_ensemble(df_ensemble_valid[good_predictors], df_scores.loc[good_predictors])
df_scores = get_scores(df_ensemble_valid, df_valid_long_wide['X_te'].var11)
df_scores.plot.bar(y='gini', figsize=(20,10), title='Model Ranking by Normalised Weighted Gini Score')

<matplotlib.axes._subplots.AxesSubplot at 0x248f59f6668>

df_scores.head(10)

The data frame df_scores shows that the top 5 models performed all better than the best teams on the public leaderboard on Kaggle.

drawing

# we simply select the best model based on its rank in the dataframe.
winner = df_scores.index[0]
print('Winner:', df_scores.index[0])

Winner: Gini Weighted Regression Ensemble

8. Final Predictions & Submission ¶

Here, we submit the final predictions.

final_predictions = predict(df_test_dict['long_wide'], df_test_dict['long_comp'], ensemble)

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:381: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, object were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)

X = final_predictions

for key, model in meta_predictors.items():
    for dataset, fit in model.fits.items():
        X = X.reindex(columns = meta_train_dict[dataset]['X_tr'].columns, fill_value=0)
        if dataset=='selected_ensemble':
            final_predictions['MetaRegressor' + '_selected_' + model.model_name] = fit.predict(X)
        else:
            final_predictions['MetaRegressor' + '_' + model.model_name] = fit.predict(X)

C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype float32, int64, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)
C:\Users\alexd\Anaconda3.6\envs\exts-aml2\lib\site-packages\sklearn\pipeline.py:331: DataConversionWarning: Data with input dtype int64, float32, float64 were all converted to float64 by StandardScaler.
  Xt = transform.transform(Xt)

final_predictions['Gini Weighted Regression Ensemble'] = gini_weighted_ensemble(final_predictions, df_scores)
final_pred = final_predictions[winner]

df_submission = pd.DataFrame()
df_submission['id'] = df_test.id
df_submission['target'] = np.exp(np.float64(final_pred))-1

df_submission.to_csv('Data\\df_submission.csv')

9. Conclusion & Discussion ¶

The following two secions conclude the capstone project and summarise the major findings.

9.1 Conclusion¶

Having gone through the entire process of preparing, cleansing, exploring, feature engineering, model building and, finally, evaluating and computing the final predictions, several observations have been made.

While we went through building a very large and complex ensemble of predictors for regression and classification, the model that ultimately was chosen as the "winning" model, was indeed the evaluation score (gini) weighted regression ensemble. However, note that the top 10 also included less complex models that used reduced data sets either by means of automatic feature selection, automatic feature compression (using PCA) or a resampled (downsampled) data set.

Following the objectives of this project, several observations have been made:

Automated feature engineering: PCA was used on three groups of features. However, there may be numerous ways how feature engineering could be automated. Models that used feature compression were fairly prominent in the top 5 in df_scores.

Automated variable selection: The function select_best_features()implements automatic feature selection based on a majority vote of different regressors and classifiers. This function appears to have been very useful and effective in the model building process as the second best model builds upon this function, which reached second place using simple linear regression.

Automated model selection: Model selection has been automated as well. Automatic model selection has been facilitated by the custom written class Model which allows for efficient and simple initialisation and training of multiple models that can be stored in a list or dictionary. Subsequently, the models can be looped through and evaluated based on the metric at hand. In this case, we used the normalized_weighted_gini()-function to evaluate model performance.

Automated ensemble building: Not only have we constructed one ensemble, we have constructed several variants of ensembles:

Simple mean of all regressors
Mean probability weighted mean of all regressors
Gini weighted mean probability weighted regressors (winner)
Majority vote weighted mean regression

Having said all that, it remains open for further disussion which model finally should be used in a productive environment. Most probably, this depends on non functional requirements such as performance (speed of prediction), explainability or maintainability. The gini weighted mean regression ensemble may have serval advantages and disadvantages.

Advantages

Accuracy
Reduced bias through averaging of multiple predictors and probabilistic features
Robustness through the inclusion of a high number of independend predictors

Disadvantages

Complex model building process
Long training time
Smaller training set due to the creation of a hold-out set that needs to be split into validation and test set.

9.2 Discussion¶

During the process of this capstone project, several learnings have happened. Hence, for a more comprehensive and extensive approach of solving the problem presented in this capstone project, the following steps may be necessary:

Parallel Computing Functions

The custom-built class Model should implement multithreaded execution of its functions as this would speed up the training process significantly.

Exhaustive Grid Search

Although the grid search has been implemented in order to find optimal model hyperparameters, the chosen parameter space was deliberately limited (to a lenght of one) in order for the models to be trained in an acceptable amount of time.

Automated feature engineering: Automated feature engineering could be extended the following ways:

An engine that automatically calculates products, ratios, reciprocals, polynomials on combinations of variables

Automatically computes a number of principal components to be used based on an explained variance threshold.

Variable Transformations

Using automatic power transformations (box-cox or yeo-johnson) of the variables might impose more symmentry and normality to the variables, which would improve learning as well. However, during the course of the project, this was not looked at. For a productive setup, however, this should be looked at more carefully.

K-Fold Cross Validation for Ensembles

The python library mlxtend implements the use of meta regressors that can use the regular sklearn API and built-in features such as grid search and cross validation. In a productive environment, a library of this sort may be beneficial. Using standard libraries icreases efficiency, maintainability as well parsimony of source code.

Extensive Feature Engineering and Subject Matter Knowledge

If the semantics of the provided features had been known to the data scientist or machine learning engineer, manual feature engineering may be a very valuable source of extra performance and accuracy of the model.

	id	var1	var2	var3	var4	var5	var6	var7	var8	var9	var10	var11	var12	var13	var14	var15	var16	var17	dummy	crimeVar1	crimeVar2	crimeVar3	crimeVar4	crimeVar5	crimeVar6	crimeVar7	crimeVar8	crimeVar9	geodemVar1	geodemVar2	geodemVar3	geodemVar4	geodemVar5	geodemVar6	geodemVar7	geodemVar8	geodemVar9	geodemVar10	geodemVar11	geodemVar12	geodemVar13	geodemVar14	geodemVar15	geodemVar16	geodemVar17	geodemVar18	geodemVar19	geodemVar20	geodemVar21	geodemVar22	geodemVar23	geodemVar24	geodemVar25	geodemVar26	geodemVar27	geodemVar28	geodemVar29	geodemVar30	geodemVar31	geodemVar32	geodemVar33	geodemVar34	geodemVar35	geodemVar36	geodemVar37	weatherVar1	weatherVar3	weatherVar4	weatherVar5	weatherVar6	weatherVar7	weatherVar8	weatherVar9	weatherVar10	weatherVar11	weatherVar12	weatherVar13	weatherVar14	weatherVar15	weatherVar16	weatherVar17	weatherVar18	weatherVar19	weatherVar20	weatherVar21	weatherVar22	weatherVar23	weatherVar24	weatherVar25	weatherVar26	weatherVar27	weatherVar28	weatherVar29	weatherVar30	weatherVar31	weatherVar32	weatherVar33	weatherVar34	weatherVar35	weatherVar36	weatherVar37	weatherVar38	weatherVar40	weatherVar42	weatherVar44	weatherVar45	weatherVar47	weatherVar48	weatherVar49	weatherVar51	weatherVar52	weatherVar53	weatherVar55	weatherVar57	weatherVar58	weatherVar59	weatherVar60	weatherVar61	weatherVar62	weatherVar63	weatherVar64	weatherVar65	weatherVar66	weatherVar67	weatherVar68	weatherVar69	weatherVar70	weatherVar71	weatherVar72	weatherVar73	weatherVar74	weatherVar75	weatherVar76	weatherVar77	weatherVar78	weatherVar79	weatherVar80	weatherVar81	weatherVar83	weatherVar84	weatherVar85	weatherVar86	weatherVar87	weatherVar88	weatherVar89	weatherVar90	weatherVar91	weatherVar92	weatherVar93	weatherVar94	weatherVar95	weatherVar96	weatherVar97	weatherVar98	weatherVar99	weatherVar100	weatherVar101	weatherVar104	weatherVar105	weatherVar106	weatherVar107	weatherVar108	weatherVar109	weatherVar110	weatherVar111	weatherVar112	weatherVar114	weatherVar116	weatherVar117	weatherVar118	weatherVar119	weatherVar120	weatherVar121	weatherVar122	weatherVar123	weatherVar124	weatherVar126	weatherVar127	weatherVar128	weatherVar130	weatherVar131	weatherVar132	weatherVar133	weatherVar134	weatherVar135	weatherVar136	weatherVar137	weatherVar138	weatherVar139	weatherVar140	weatherVar141	weatherVar142	weatherVar143	weatherVar144	weatherVar146	weatherVar148	weatherVar150	weatherVar151	weatherVar153	weatherVar154	weatherVar155	weatherVar157	weatherVar158	weatherVar159	weatherVar161	weatherVar163	weatherVar164	weatherVar166	weatherVar168	weatherVar169	weatherVar170	weatherVar171	weatherVar172	weatherVar173	weatherVar174	weatherVar175	weatherVar176	weatherVar177	weatherVar178	weatherVar179	weatherVar180	weatherVar181	weatherVar182	weatherVar183	weatherVar184	weatherVar185	weatherVar186	weatherVar187	weatherVar188	weatherVar189	weatherVar190	weatherVar191	weatherVar192	weatherVar193	weatherVar194	weatherVar195	weatherVar196	weatherVar197	weatherVar198	weatherVar199	weatherVar200	weatherVar203	weatherVar204	weatherVar205	weatherVar206	weatherVar207	weatherVar208	weatherVar209	weatherVar211	weatherVar212	weatherVar213	weatherVar214	weatherVar215	weatherVar216	weatherVar217	weatherVar218	weatherVar219	weatherVar221	weatherVar222	weatherVar223	weatherVar225	weatherVar226	weatherVar227	weatherVar229	weatherVar230	weatherVar231	weatherVar232	weatherVar233	weatherVar234	weatherVar235	weatherVar236
0	1	NaN	NaN	NaN	N1	NaN	NaN	3.0	1.0	A	3.898949	705.907218	NaN	0.000000	0.0	29.0	NaN	84.337349	A	0.868651	0.625186	0.685535	0.698889	1.322650	1.036953	0.739831	1.233004	0.985852	2.907669	1.190971	1.085760	1.214415	4.186556	2.711295	0.902699	2.682806	1.036433	2.624847	2.632027	0.530775	1.934022	0.702882	1.957826	0.977705	2.430076	0.642147	1.351742	2.838848	2.869259	1.152205	1.261544	1.468264	1.769648	2.646861	3.034098	2.868507	1.333730	0.780628	0.713642	2.417470	3.029365	1.091405	2.985905	4.261874	0.959292	0.182784	1.126181	0.001438	0.291717	0.005983	1.103744	1.444520	0.000000	1.016891	0.147343	1.967861	0.022343	1.111517	1.150924	1.011941	0.002227	0.282293	0.006597	0.381844	0.674719	0.125078	0.148698	0.161928	0.002323	0.503055	1.011204	0.144197	1.228219	1.004391	0.009624	1.104245	0.852247	0.347657	0.811226	0.679922	0.002481	0.169487	1.090833	0.253126	1.135944	0.817920	1.073134	0.178608	2.212616	1.061358	1.137966	0.905694	0.253435	0.407403	0.611287	0.614592	0.152828	0.071092	0.002797	0.735235	1.012214	0.147134	5.586985	4.683603	0.058903	1.093778	0.949523	0.348595	0.934886	0.665584	0.002785	0.153111	1.579249	0.007837	0.724788	0.109090	1.524264	3.870283	0.926208	0.168710	1.592272	0.126252	1.528397	1.677643	0.892934	0.009734	0.723420	0.052464	0.771335	0.789749	0.000000	0.754240	0.136047	0.000000	0.000000	1.012377	0.785233	0.008502	1.503067	0.636448	0.468449	1.517885	0.553574	0.000596	0.000000	0.959892	0.015623	0.945053	0.000000	0.000000	0.986450	0.000000	1.804190	0.959318	0.967749	0.902594	0.015616	0.071522	0.452953	0.062356	0.000000	0.000000	0.235065	1.009435	0.061089	0.000000	0.000000	19.504291	0.950587	0.956064	0.236168	0.475049	0.855036	0.235344	1.151152	0.275009	1.107113	4.767955	1.174688	0.246032	2.696544	1.130599	1.184083	1.209877	0.256026	0.390436	0.757689	0.390010	0.000000	0.000000	1.008965	0.387164	17.763228	16.580859	0.050419	1.117644	0.805361	0.311635	0.945178	0.643006	0.00000	0.933556	1.513327	0.508434	0.987788	0.864850	0.975799	1.032288	1.014591	0.000000	1.158028	1.275393	0.911490	0.987212	1.193584	0.929662	0.959245	1.433812	0.884953	0.050891	0.087160	0.000000	0.834392	0.037845	0.000000	0.000000	0.000000	1.534205	0.000000	0.101716	0.359774	0.947338	0.573140	0.340894	1.426992	0.000000	0.000000	0.805522	0.000000	0.000000	1.380741	1.488921	0.170351	0.00000	0.000000	1.117353	1.215303	0.112556	2.355737	0.404655	0.138667
1	2	NaN	NaN	NaN	C1	NaN	NaN	3.0	2.0	A	3.898949	778.939664	NaN	1.268636	NaN	23.0	NaN	100.000000	A	1.049803	0.668872	0.902722	0.658104	1.679865	0.505010	0.754489	0.887022	0.977406	3.291752	1.108728	1.009916	1.163157	4.376695	2.370791	0.894850	2.690246	1.003267	2.569522	2.655512	1.844761	3.520036	1.283554	4.074402	1.015829	2.584384	3.417460	1.134306	1.827247	2.556867	1.022824	1.110723	0.978843	0.328712	2.616680	2.641717	2.357490	3.229030	0.993527	0.962851	3.214011	2.677525	0.975244	2.558843	1.214572	1.033903	0.096741	1.103729	0.001916	0.284243	0.007736	1.112626	0.841267	0.000000	1.054624	0.085023	1.888556	0.008875	1.085206	1.132958	1.137549	0.002967	0.269912	0.008531	0.413463	0.747089	0.171156	0.120307	0.054054	0.001896	0.861806	1.010997	0.113343	1.973173	1.613586	0.016845	1.098581	0.839476	0.365417	1.208152	0.662309	0.001956	0.111328	1.069970	0.263695	1.149855	0.357706	1.123981	0.117320	2.154383	1.038734	1.120017	1.055427	0.264093	0.492971	0.732531	0.666414	0.103689	0.000000	0.000000	0.929382	1.011841	0.095660	2.039497	1.709723	0.151014	1.090714	0.928610	0.347350	1.353240	0.644170	0.000000	0.142694	1.552106	0.010189	0.624482	0.139304	1.543782	1.755138	0.938527	0.157232	1.467947	0.050672	1.511962	1.631539	0.982163	0.012656	0.619290	0.066995	0.797095	0.842630	0.000000	0.546071	0.000000	0.000000	0.000000	1.012136	0.566820	0.011553	1.491088	0.627510	0.411904	1.830869	0.544481	0.000239	0.000000	0.941338	0.015322	0.952575	0.150204	0.000000	1.064851	0.000000	1.807509	0.928614	0.960588	1.075041	0.015190	0.088404	0.539412	0.050584	0.000000	0.000000	0.539235	1.009411	0.047463	0.000000	0.000000	50.350685	0.947018	0.938100	0.312246	0.792025	0.796838	0.085970	1.128644	0.291433	1.113917	2.944287	1.194877	0.089874	2.547686	1.107284	1.161393	1.397612	0.265442	0.415435	0.850667	0.285679	0.000000	0.000000	1.009171	0.270114	95.551062	89.190923	0.000000	1.112011	0.794264	0.324268	1.237568	0.631341	0.00000	1.004587	1.641960	0.510606	1.023067	0.855347	0.985656	1.032288	0.795942	0.001693	1.163602	1.465508	1.015864	1.133323	1.396598	1.074653	1.051767	1.591731	0.934491	0.390163	0.087160	0.005752	0.955027	0.054529	0.367790	0.128270	0.001751	1.342430	0.000000	0.000000	0.279998	0.197783	0.211157	0.218776	0.234640	0.000000	0.000000	0.296771	0.000000	0.000000	0.460247	0.271985	0.266173	27.50823	0.000000	3.828979	1.036739	0.033052	0.856632	0.231232	0.742199
2	3	3.0	NaN	4.0	J3	B	B	2.0	4.0	NaN	4.314082	686.728476	2.719846	2.482178	0.0	NaN	100.0	0.000000	B	1.887331	1.513871	1.274002	1.256733	1.046967	1.070620	1.302359	1.122016	1.572879	2.437528	0.942239	1.061809	1.029098	2.927603	0.911176	0.753558	0.807826	1.015704	0.926616	0.944779	0.826123	0.944326	0.963204	0.869176	1.005991	0.873743	0.738184	0.914992	0.506094	1.019649	0.933150	0.882553	0.978843	0.484727	0.833541	1.110557	1.268557	2.831251	1.080263	2.401463	2.546548	1.051406	1.173942	0.865429	1.149648	1.119174	0.245670	1.291792	0.000000	1.373878	0.000000	1.394500	0.000000	0.731094	1.083278	0.262800	0.633516	0.012947	1.246473	1.360232	0.723947	0.000000	1.377227	0.000000	1.119032	1.013013	0.024569	2.071703	0.388993	0.006210	1.318338	1.024891	2.081796	0.331280	0.406362	0.008509	1.309766	1.113131	0.994076	0.871906	0.973890	0.006288	0.607247	1.302429	0.632562	1.413798	0.000000	1.070749	0.639929	0.791548	1.259822	1.368421	0.598745	0.634719	0.554160	0.621567	0.000000	1.024714	1.149162	0.048348	1.770925	1.025741	1.021700	0.000000	0.000000	0.000000	1.309314	1.049828	0.679810	0.358934	0.948085	0.048140	0.000000	1.691054	0.000000	0.843731	0.000000	1.921345	0.000000	1.107522	0.000000	0.511728	0.072227	1.607050	1.831677	0.918046	0.000000	0.846533	0.000000	0.871676	0.874009	0.000000	1.810339	0.000000	0.000000	0.172094	1.026003	1.901328	0.030361	1.697237	1.025580	0.686021	0.197254	0.911006	0.000341	1.330801	1.094529	2.251127	1.162168	0.000000	3.101600	1.084263	1.410383	0.584333	1.066076	1.134036	0.671377	2.264050	1.844322	1.469363	2.526268	1.369190	2.383756	0.933704	1.024431	2.527229	0.000000	0.559062	0.000000	1.118664	1.147422	1.616393	1.220836	0.903446	0.000000	1.305617	1.112534	1.415131	0.000000	0.977427	0.000000	0.482193	1.236363	1.406393	0.600480	1.114737	0.874057	0.798642	1.380095	0.000000	0.589675	1.022330	1.389399	0.000000	0.000000	0.000000	1.327500	1.138879	0.913905	0.805358	1.063889	0.00000	1.126355	3.163799	0.123604	3.298508	0.883858	0.837807	0.853623	1.647727	0.000000	1.345148	1.634286	1.089802	1.208149	1.526846	1.049443	1.309290	1.843008	1.146110	1.441908	0.615548	0.003097	0.975133	0.938318	1.011424	1.154427	0.003501	1.054766	32.766780	11.392218	1.497997	2.790136	1.101031	0.635485	0.858505	1.235534	4.971308	0.763126	1.460080	4.942563	0.920494	3.014886	0.979517	0.00000	0.456134	0.098790	1.076535	0.566352	0.696013	0.693695	0.070654
3	4	3.0	NaN	4.0	H1	B	NaN	3.0	4.0	NaN	4.314082	3515.206111	3.520478	2.054144	0.0	22.0	100.0	100.000000	B	1.659096	1.341626	0.957322	1.328928	0.661204	1.232223	1.276299	0.904233	1.346898	0.301106	0.767103	0.870204	1.080356	0.920084	0.197981	2.394116	0.872650	0.671609	0.207778	0.321540	0.370501	0.371143	1.091631	0.189601	1.071171	0.389568	0.464722	0.884784	0.076090	0.402487	1.263741	1.388941	0.978843	0.279160	0.303119	0.450161	0.270244	-3.463018	0.780628	1.517906	0.586644	0.375894	1.165983	0.421828	0.055546	1.193785	0.083387	1.302432	0.000000	1.307082	0.000000	1.427265	0.043718	15.233289	0.868432	0.089202	0.526142	0.000000	1.243298	1.389218	0.640472	0.000000	1.309043	0.000000	0.966190	0.877593	0.094990	1.942914	0.042947	0.001313	0.200095	1.026034	1.956164	6.844918	5.890540	0.002019	1.330546	1.087766	0.828558	0.878915	1.053989	0.001309	0.068621	1.306121	0.641005	1.470076	0.000000	0.807000	0.072314	0.693332	1.245927	1.400001	0.621100	0.645153	0.395751	0.479376	0.000000	0.724476	0.000000	0.011339	0.000000	1.027527	0.730578	0.000000	0.000000	0.000000	1.339040	1.052583	0.569376	0.654234	1.003897	0.011291	0.154459	1.717281	0.000000	0.819391	0.000000	2.012987	0.000000	1.047249	0.170196	0.512281	0.000000	1.612206	1.896058	0.936382	0.000000	0.821213	0.000000	0.823743	0.825240	0.000000	1.054685	0.000000	0.000000	0.808274	1.027961	1.101187	0.000000	1.744952	1.015777	0.694476	0.398237	0.963261	0.000000	0.310827	1.093205	2.318051	1.172217	0.175902	6.745614	0.790483	0.329415	0.449436	1.049831	1.150550	0.410760	2.362104	1.684506	1.308318	2.586542	0.310381	0.286573	0.079508	1.027137	2.592049	13.407909	12.209011	0.000000	1.125760	1.073781	1.202004	1.101790	1.005862	0.000000	1.323631	0.788416	1.437979	0.000000	0.763923	0.000000	0.506057	1.249600	1.435646	0.508266	0.789722	0.714955	0.651096	1.346044	0.000000	0.570324	1.023985	1.358656	0.000000	0.000000	0.000000	1.347419	1.127120	0.856942	1.109113	1.157202	0.00000	1.481512	2.570764	0.236545	0.352782	0.817331	0.827951	0.813920	1.341776	0.000000	1.284199	1.947960	1.091822	1.261574	1.750477	1.048165	1.457046	2.330699	1.153004	0.152673	0.615548	0.001327	0.924869	0.161473	1.011424	0.128270	0.000175	3.068410	0.000000	0.000000	0.362714	0.418882	0.135744	0.331984	4.059021	0.000000	0.000000	0.190781	0.000000	0.000000	0.690371	1.316712	0.308761	0.00000	4.349042	0.401975	0.340631	0.290147	0.000000	0.000000	0.090332
4	5	NaN	NaN	NaN	H1	NaN	NaN	2.0	4.0	A	3.898949	1014.569367	NaN	0.832555	NaN	21.0	NaN	2.421308	A	0.894055	1.380088	1.133255	1.249629	1.258417	0.922484	1.285127	1.022282	0.954293	1.259461	1.059423	1.029875	1.048813	3.182098	1.209454	0.682912	1.206430	1.173242	1.762718	1.556458	0.937147	3.762755	2.352705	1.247714	0.999842	1.387093	0.702374	1.015946	2.150686	1.504036	1.005960	0.963427	0.978843	0.821780	1.344888	1.241862	1.351868	1.661313	0.954101	2.695982	4.603116	1.517947	1.049912	1.247144	1.278466	0.820727	0.538739	1.178029	0.081550	0.964872	0.167968	1.177430	0.748524	0.000000	0.693000	0.525132	0.623646	0.271918	1.169077	1.204121	0.574390	0.011114	0.965669	0.033263	0.817763	0.738407	0.440639	1.327029	0.337209	0.145860	1.468817	1.004333	1.331123	0.313348	0.256244	0.369253	1.163302	1.108650	1.422075	1.196323	1.252426	0.146830	0.920142	1.202517	0.820090	1.281154	1.152767	0.841680	0.890741	1.087205	1.176175	1.245454	0.839048	0.822724	0.694339	0.752309	0.000000	1.647610	0.469290	0.062614	3.385186	1.003111	1.645250	0.000000	0.000000	0.013418	1.213830	1.028244	1.450071	1.822340	1.279203	0.062345	0.426891	1.329093	0.036082	1.065323	0.261988	1.323293	4.893752	0.785975	0.325946	0.642832	0.426048	1.316101	1.350939	0.565869	0.004829	1.069832	0.026000	0.933586	0.758901	0.346244	2.224640	0.542020	0.102242	0.571744	1.006082	2.321656	0.485092	1.296256	1.113285	1.274797	0.797303	1.158940	0.104081	0.258792	1.127015	0.589839	1.087739	0.150165	0.000000	0.317519	0.274268	0.258526	1.115442	1.141762	0.294686	0.567897	0.567153	0.472456	0.808865	0.000000	0.489287	1.279601	1.004007	0.804131	0.000000	0.000000	0.000000	1.091278	1.149158	1.431500	1.180554	1.273779	0.443130	1.166772	1.265141	1.160478	0.000000	0.821812	0.463254	0.720823	1.154640	1.183226	0.627362	1.261508	0.841530	0.784844	2.225859	0.100143	4.813847	1.003796	2.220833	0.000000	0.000000	0.048324	1.150822	1.123673	1.377604	1.176790	1.209113	0.09951	0.801640	2.440240	0.489281	1.322931	0.921874	0.936373	1.260583	0.620552	0.088445	1.210986	1.294181	1.119565	1.179820	1.318982	1.126934	1.134967	1.248954	1.110199	3.748960	4.425052	2.467910	1.196297	2.552121	2.114795	6.541753	7.193384	2.876635	1.287243	1.645525	0.768681	0.552920	1.101031	0.824124	2.449361	1.578738	2.761838	0.847917	5.110279	7.413845	2.761483	1.629389	2.171972	0.00000	0.514990	0.516095	1.016120	1.313732	1.338487	2.948202	0.816485

	LinearRegression_long_wide_reg	LinearRegression_short_wide_reg	LinearRegression_long_comp_reg	LinearRegression_short_comp_reg	LinearRegression_long_selected_reg	LinearRegression_short_selected_reg	Lasso_long_wide_reg	Lasso_short_wide_reg	Lasso_long_comp_reg	Lasso_short_comp_reg	Lasso_long_selected_reg	Lasso_short_selected_reg	Ridge_long_wide_reg	Ridge_short_wide_reg	Ridge_long_comp_reg	Ridge_short_comp_reg	Ridge_long_selected_reg	Ridge_short_selected_reg	RandomForestRegression_long_comp_reg	RandomForestRegression_short_comp_reg	RandomForestRegression_long_selected_reg	RandomForestRegression_short_selected_reg	GradientBoostingRegression_long_comp_reg	GradientBoostingRegression_short_comp_reg	GradientBoostingRegression_long_selected_reg	GradientBoostingRegression_short_selected_reg	XGBRegression_long_comp_reg	XGBRegression_short_comp_reg	XGBRegression_long_selected_reg	XGBRegression_short_selected_reg	LinearSVR_long_comp_reg	LinearSVR_short_comp_reg	LinearSVR_long_selected_reg	LinearSVR_short_selected_reg	LogisticRegression_long_wide_clf	LogisticRegression_long_wide_prob	LogisticRegression_long_comp_clf	LogisticRegression_long_comp_prob	LogisticRegression_long_selected_clf	LogisticRegression_long_selected_prob	ExtraTreesClassifier_long_wide_prob	RandomForestClassifier_long_comp_prob	RandomForestClassifier_long_selected_prob	Regressor Mean	Probability Mean	Mean Probability Weighted Regression Mean
0	0.000768	0.017345	0.000454	0.015511	0.000603	0.030339	0.001679	0.044245	0.000593	0.018158	0.000696	0.026978	0.001041	0.020487	0.000626	0.021655	0.000939	0.034857	0.000840	0.027703	0.001041	0.039390	0.001797	0.061505	0.001845	0.061607	0.001284	0.012597	0.001425	0.057300	9.828482e-12	-0.000831	-0.000004	0.005784	0	0.146780	0	0.134564	0	0.183567	0.000000	0.000000	0.000000	0.015008	0.051657	0.000775
1	0.004801	0.154033	0.004102	0.131692	0.004929	0.162782	0.003257	0.130926	0.004134	0.132942	0.004467	0.159577	0.004837	0.133897	0.003947	0.122592	0.004156	0.137183	0.003103	0.157558	0.003485	0.141384	0.002758	0.160537	0.002917	0.109189	0.003050	0.254423	0.003250	0.213258	-1.915372e-11	-0.000229	0.000001	-0.000233	1	0.560558	1	0.526242	1	0.572146	0.003333	0.003333	0.013333	0.069374	0.186550	0.012942
2	0.002108	0.063656	0.000311	0.015023	0.000201	0.001374	0.001765	0.044047	0.000273	0.015067	0.000435	0.006519	0.001346	0.055392	0.000443	0.021107	0.000573	0.018712	0.001013	0.032137	0.001092	0.031556	0.001865	0.062193	0.001932	0.062164	0.001292	0.070604	0.001220	0.031517	-6.996481e-12	-0.000451	0.000007	-0.011092	0	0.203778	0	0.171000	0	0.150766	0.000000	0.006667	0.000000	0.015747	0.059135	0.000931
3	-0.000791	-0.042553	0.000209	-0.005515	0.000771	0.015950	0.001854	0.052359	0.000675	0.002497	0.001020	0.022473	0.000638	0.032528	0.001112	0.036017	0.001446	0.042219	0.000985	0.036830	0.001119	0.038634	0.001855	0.061530	0.001932	0.061607	0.001284	0.035784	0.001508	0.081373	4.336611e-12	-0.000965	0.000005	-0.002550	0	0.093466	0	0.202043	0	0.159150	0.000000	0.000000	0.000000	0.014231	0.050518	0.000719
4	0.004938	0.112575	0.003360	0.095742	0.004042	0.127885	0.002436	0.081264	0.003252	0.094652	0.003783	0.122422	0.004457	0.116454	0.003120	0.090484	0.003576	0.112784	0.003845	0.089389	0.003098	0.089343	0.003173	0.086034	0.002784	0.082657	0.003282	0.100879	0.003164	0.073896	-1.334434e-11	0.001273	0.000004	0.011612	1	0.704560	1	0.566494	1	0.652269	0.000000	0.006667	0.000000	0.045343	0.214443	0.009723

	gini	model type	weight
Gini Weighted Regression Ensemble	0.472582	Ensemble	0.073910
Gini Weighted Regression Ensemble (Selected)	0.469600	Ensemble	0.073444
Mean Probability Weighted Regression Mean	0.466181	Ensemble	0.072909
MetaRegressor_Ridge	0.465821	Ensemble	0.072853
Lasso_short_selected_reg	0.465178	Regressor	0.037515
MetaRegressor_selected_ElasticNet	0.465070	Ensemble	0.072735
Lasso_short_comp_reg	0.463569	Regressor	0.037385
Ridge_short_comp_reg	0.462702	Regressor	0.037315
Ridge_short_selected_reg	0.462514	Regressor	0.037300
LinearRegression_short_comp_reg	0.461805	Regressor	0.037243

	Count	dType	nulls	nulls pct	not nulls	duplicates	duplicates pct	first value
var2	4440	object	447621	0.990178	4440	4437	0.009815	A
var6	26850	object	425211	0.940605	26850	26847	0.059388	B
var5	36145	object	415916	0.920044	36145	36139	0.079943	B
var3	57027	float64	395034	0.873851	57027	57021	0.126136	4
var16	90368	float64	361693	0.800098	90368	90268	0.199681	100
var12	97019	float64	355042	0.785385	97019	76973	0.170271	2.71985
var1	114096	float64	337965	0.747609	114096	114091	0.252380	3
var14	161595	float64	290466	0.642537	161595	161531	0.357321	0
var9	328876	object	123185	0.272496	328876	328874	0.727499	A
crimeVar7	334698	float64	117363	0.259618	334698	334379	0.739677	0.739831
crimeVar2	337508	float64	114553	0.253402	337508	337187	0.745888	0.625186
crimeVar4	339263	float64	112798	0.249519	339263	338945	0.749777	0.698889
crimeVar5	341406	float64	110655	0.244779	341406	341094	0.754531	1.32265
crimeVar6	342073	float64	109988	0.243303	342073	341963	0.756453	1.03695
crimeVar1	342073	float64	109988	0.243303	342073	341762	0.756009	0.868651
crimeVar9	342073	float64	109988	0.243303	342073	341754	0.755991	0.985852
crimeVar8	342073	float64	109988	0.243303	342073	341773	0.756033	1.233
crimeVar3	342073	float64	109988	0.243303	342073	341838	0.756177	0.685535
var15	353205	float64	98856	0.218678	353205	352971	0.780804	29
weatherVar212	419688	float64	32373	0.071612	419688	419250	0.927419	0.101716
weatherVar218	419688	float64	32373	0.071612	419688	419629	0.928257	0
weatherVar211	419688	float64	32373	0.071612	419688	419558	0.928100	0
weatherVar214	419688	float64	32373	0.071612	419688	418013	0.924683	0.947338
weatherVar213	419688	float64	32373	0.071612	419688	417891	0.924413	0.359774
weatherVar209	419688	float64	32373	0.071612	419688	419631	0.928262	1.53421
weatherVar210	419688	float64	32373	0.071612	419688	419589	0.928169	0
weatherVar217	419688	float64	32373	0.071612	419688	417423	0.923378	1.42699
weatherVar220	419688	float64	32373	0.071612	419688	419663	0.928333	0
weatherVar219	419688	float64	32373	0.071612	419688	419652	0.928308	0
weatherVar216	419688	float64	32373	0.071612	419688	418266	0.925242	0.340894
weatherVar221	419688	float64	32373	0.071612	419688	419599	0.928191	0.805522
weatherVar222	419688	float64	32373	0.071612	419688	419674	0.928357	0
weatherVar223	419688	float64	32373	0.071612	419688	419681	0.928372	0
weatherVar224	419688	float64	32373	0.071612	419688	419683	0.928377	0
weatherVar225	419688	float64	32373	0.071612	419688	419660	0.928326	1.38074
weatherVar226	419688	float64	32373	0.071612	419688	418198	0.925092	1.48892
weatherVar215	419688	float64	32373	0.071612	419688	419545	0.928072	0.57314
var4	431759	object	20302	0.044910	431759	431717	0.954997	N1
weatherVar206	432675	float64	19386	0.042884	432675	432616	0.956986	0
weatherVar208	432675	float64	19386	0.042884	432675	432100	0.955844	0
weatherVar207	432675	float64	19386	0.042884	432675	432610	0.956973	0
weatherVar205	432675	float64	19386	0.042884	432675	430165	0.951564	0.0378446
weatherVar204	432675	float64	19386	0.042884	432675	432585	0.956917	0.834392
weatherVar203	432675	float64	19386	0.042884	432675	432274	0.956229	0
weatherVar202	432675	float64	19386	0.042884	432675	432610	0.956973	0
weatherVar201	432675	float64	19386	0.042884	432675	432543	0.956824	0
weatherVar200	432675	float64	19386	0.042884	432675	432627	0.957010	0.0871601
weatherVar199	432675	float64	19386	0.042884	432675	432463	0.956647	0.0508909
weatherVar229	432706	float64	19355	0.042815	432706	432592	0.956933	0
weatherVar227	432706	float64	19355	0.042815	432706	432510	0.956751	0.170351
weatherVar228	432706	float64	19355	0.042815	432706	432625	0.957006	0
weatherVar236	432706	float64	19355	0.042815	432706	431136	0.953712	0.138667
weatherVar232	432706	float64	19355	0.042815	432706	431086	0.953601	1.2153
weatherVar231	432706	float64	19355	0.042815	432706	432094	0.955831	1.11735
weatherVar230	432706	float64	19355	0.042815	432706	432313	0.956316	0
weatherVar234	432706	float64	19355	0.042815	432706	432635	0.957028	2.35574
weatherVar235	432706	float64	19355	0.042815	432706	432634	0.957026	0.404655
weatherVar233	432706	float64	19355	0.042815	432706	430190	0.951619	0.112556
weatherVar192	439962	float64	12099	0.026764	439962	437553	0.967907	0.91149
weatherVar187	439962	float64	12099	0.026764	439962	439717	0.972694	1.03229
weatherVar188	439962	float64	12099	0.026764	439962	435234	0.962777	1.01459
weatherVar189	439962	float64	12099	0.026764	439962	434002	0.960052	0
weatherVar190	439962	float64	12099	0.026764	439962	436565	0.965721	1.15803
weatherVar191	439962	float64	12099	0.026764	439962	435278	0.962874	1.27539
weatherVar194	439962	float64	12099	0.026764	439962	435200	0.962702	1.19358
weatherVar193	439962	float64	12099	0.026764	439962	436563	0.965717	0.987212
weatherVar184	439962	float64	12099	0.026764	439962	439311	0.971796	0.987788
weatherVar195	439962	float64	12099	0.026764	439962	437450	0.967679	0.929662
weatherVar196	439962	float64	12099	0.026764	439962	436430	0.965423	0.959245
weatherVar197	439962	float64	12099	0.026764	439962	435288	0.962897	1.43381
weatherVar198	439962	float64	12099	0.026764	439962	437197	0.967119	0.884953
weatherVar185	439962	float64	12099	0.026764	439962	439723	0.972707	0.86485
weatherVar186	439962	float64	12099	0.026764	439962	439865	0.973021	0.975799
weatherVar182	439962	float64	12099	0.026764	439962	436793	0.966226	1.51333
weatherVar181	439962	float64	12099	0.026764	439962	439793	0.972862	0.933556
weatherVar183	439962	float64	12099	0.026764	439962	432957	0.957740	0.508434
var8	444606	float64	7455	0.016491	444606	444600	0.983496	1
weatherVar115	448201	float64	3860	0.008539	448201	448200	0.991459	0
weatherVar79	448246	float64	3815	0.008439	448246	435590	0.963565	0.10909
weatherVar92	448246	float64	3815	0.008439	448246	437886	0.968644	0.0524644
weatherVar56	448289	float64	3772	0.008344	448289	439467	0.972141	0
weatherVar43	448289	float64	3772	0.008344	448289	438223	0.969389	0
weatherVar162	448401	float64	3660	0.008096	448401	439943	0.973194	0
weatherVar149	448401	float64	3660	0.008096	448401	439262	0.971687	0
weatherVar6	448634	float64	3427	0.007581	448634	434683	0.961558	0.00598288
weatherVar19	448634	float64	3427	0.007581	448634	437339	0.967434	0.00659739
weatherVar41	449481	float64	2580	0.005707	449481	440687	0.974840	0
weatherVar54	449481	float64	2580	0.005707	449481	441917	0.977561	0
weatherVar90	449741	float64	2320	0.005132	449741	439396	0.971984	0.00973423
weatherVar77	449741	float64	2320	0.005132	449741	438860	0.970798	0.00783658
weatherVar160	449776	float64	2285	0.005055	449776	443301	0.980622	0
weatherVar147	449776	float64	2285	0.005055	449776	442959	0.979866	0
weatherVar113	449854	float64	2207	0.004882	449854	448999	0.993227	0
weatherVar125	449854	float64	2207	0.004882	449854	449027	0.993289	0
weatherVar17	449989	float64	2072	0.004583	449989	438802	0.970670	0.00222676
weatherVar4	449989	float64	2072	0.004583	449989	436133	0.964766	0.00143787
geodemVar25	450065	float64	1996	0.004415	450065	449508	0.994353	1.76965
geodemVar24	450065	float64	1996	0.004415	450065	450062	0.995578	1.46826
geodemVar35	450065	float64	1996	0.004415	450065	447832	0.990645	2.9859
geodemVar27	450065	float64	1996	0.004415	450065	448343	0.991775	3.0341
geodemVar20	450065	float64	1996	0.004415	450065	448335	0.991758	2.83885
geodemVar21	450065	float64	1996	0.004415	450065	447889	0.990771	2.86926
geodemVar22	450065	float64	1996	0.004415	450065	447739	0.990439	1.15221
geodemVar23	450065	float64	1996	0.004415	450065	447675	0.990298	1.26154
geodemVar26	450065	float64	1996	0.004415	450065	447510	0.989933	2.64686
geodemVar31	450065	float64	1996	0.004415	450065	449530	0.994401	0.713642
geodemVar28	450065	float64	1996	0.004415	450065	447583	0.990094	2.86851
geodemVar18	450065	float64	1996	0.004415	450065	448847	0.992890	0.642147
geodemVar36	450065	float64	1996	0.004415	450065	448346	0.991782	4.26187
geodemVar1	450065	float64	1996	0.004415	450065	447627	0.990192	2.90767
geodemVar34	450065	float64	1996	0.004415	450065	447727	0.990413	1.0914
geodemVar33	450065	float64	1996	0.004415	450065	447491	0.989891	3.02936
geodemVar29	450065	float64	1996	0.004415	450065	449314	0.993923	1.33373
geodemVar30	450065	float64	1996	0.004415	450065	449776	0.994945	0.780628
geodemVar19	450065	float64	1996	0.004415	450065	447577	0.990081	1.35174
geodemVar32	450065	float64	1996	0.004415	450065	447783	0.990537	2.41747
geodemVar17	450065	float64	1996	0.004415	450065	447570	0.990066	2.43008
geodemVar3	450065	float64	1996	0.004415	450065	449785	0.994965	1.08576
geodemVar5	450065	float64	1996	0.004415	450065	447876	0.990742	4.18656
geodemVar4	450065	float64	1996	0.004415	450065	449697	0.994771	1.21441
geodemVar16	450065	float64	1996	0.004415	450065	449666	0.994702	0.977705
geodemVar8	450065	float64	1996	0.004415	450065	447681	0.990311	2.68281
geodemVar7	450065	float64	1996	0.004415	450065	449841	0.995089	0.902699
geodemVar10	450065	float64	1996	0.004415	450065	447616	0.990167	2.62485
geodemVar9	450065	float64	1996	0.004415	450065	449866	0.995144	1.03643
geodemVar11	450065	float64	1996	0.004415	450065	447508	0.989928	2.63203
geodemVar2	450065	float64	1996	0.004415	450065	447618	0.990172	1.19097
geodemVar12	450065	float64	1996	0.004415	450065	447541	0.990001	0.530775
geodemVar13	450065	float64	1996	0.004415	450065	447485	0.989877	1.93402
geodemVar14	450065	float64	1996	0.004415	450065	447793	0.990559	0.702882
geodemVar15	450065	float64	1996	0.004415	450065	447846	0.990676	1.95783
geodemVar6	450065	float64	1996	0.004415	450065	447508	0.989928	2.71129
geodemVar37	450066	float64	1995	0.004413	450066	449896	0.995211	0.959292
weatherVar148	450201	float64	1860	0.004114	450201	432752	0.957287	0.275009
weatherVar146	450201	float64	1860	0.004114	450201	437199	0.967124	1.15115
weatherVar145	450201	float64	1860	0.004114	450201	450151	0.995775	0
weatherVar144	450201	float64	1860	0.004114	450201	440856	0.975214	0.235344
weatherVar143	450201	float64	1860	0.004114	450201	437245	0.967226	0.855036
weatherVar142	450201	float64	1860	0.004114	450201	433929	0.959890	0.475049
weatherVar141	450201	float64	1860	0.004114	450201	437057	0.966810	0.236168
weatherVar140	450201	float64	1860	0.004114	450201	436968	0.966613	0.956064
weatherVar135	450201	float64	1860	0.004114	450201	437153	0.967022	0.0610891
weatherVar139	450201	float64	1860	0.004114	450201	437292	0.967330	0.950587
weatherVar138	450201	float64	1860	0.004114	450201	449176	0.993618	19.5043
weatherVar137	450201	float64	1860	0.004114	450201	447320	0.989512	0
weatherVar136	450201	float64	1860	0.004114	450201	447757	0.990479	0
weatherVar134	450201	float64	1860	0.004114	450201	437033	0.966757	1.00944
weatherVar133	450201	float64	1860	0.004114	450201	435148	0.962587	0.235065
weatherVar132	450201	float64	1860	0.004114	450201	438722	0.970493	0
weatherVar131	450201	float64	1860	0.004114	450201	441493	0.976623	0
weatherVar130	450201	float64	1860	0.004114	450201	437177	0.967075	0.0623563
weatherVar129	450201	float64	1860	0.004114	450201	448535	0.992200	0
weatherVar127	450201	float64	1860	0.004114	450201	437097	0.966898	0.0715222
weatherVar126	450201	float64	1860	0.004114	450201	436962	0.966600	0.0156157
weatherVar124	450201	float64	1860	0.004114	450201	437194	0.967113	0.902594
weatherVar128	450201	float64	1860	0.004114	450201	436713	0.966049	0.452953
weatherVar122	450201	float64	1860	0.004114	450201	437179	0.967080	0.959318
weatherVar150	450201	float64	1860	0.004114	450201	437182	0.967086	1.10711
weatherVar151	450201	float64	1860	0.004114	450201	446289	0.987232	4.76796
weatherVar180	450201	float64	1860	0.004114	450201	437156	0.967029	0
weatherVar179	450201	float64	1860	0.004114	450201	437248	0.967232	0.643006
weatherVar178	450201	float64	1860	0.004114	450201	433585	0.959129	0.945178
weatherVar177	450201	float64	1860	0.004114	450201	436985	0.966651	0.311635
weatherVar176	450201	float64	1860	0.004114	450201	436912	0.966489	0.805361
weatherVar175	450201	float64	1860	0.004114	450201	437213	0.967155	1.11764
weatherVar174	450201	float64	1860	0.004114	450201	438131	0.969186	0.0504188
weatherVar173	450201	float64	1860	0.004114	450201	449477	0.994284	16.5809
weatherVar172	450201	float64	1860	0.004114	450201	449496	0.994326	17.7632
weatherVar171	450201	float64	1860	0.004114	450201	436438	0.965441	0.387164
weatherVar170	450201	float64	1860	0.004114	450201	437045	0.966783	1.00896
weatherVar169	450201	float64	1860	0.004114	450201	440242	0.973855	0
weatherVar168	450201	float64	1860	0.004114	450201	437139	0.966991	0
weatherVar167	450201	float64	1860	0.004114	450201	444351	0.982945	0
weatherVar166	450201	float64	1860	0.004114	450201	436509	0.965598	0.39001
weatherVar165	450201	float64	1860	0.004114	450201	446892	0.988566	0
weatherVar164	450201	float64	1860	0.004114	450201	436728	0.966082	0.757689
weatherVar163	450201	float64	1860	0.004114	450201	437126	0.966962	0.390436
weatherVar161	450201	float64	1860	0.004114	450201	436950	0.966573	0.256026
weatherVar159	450201	float64	1860	0.004114	450201	437187	0.967097	1.20988
weatherVar158	450201	float64	1860	0.004114	450201	437107	0.966920	1.18408
weatherVar157	450201	float64	1860	0.004114	450201	437204	0.967135	1.1306
weatherVar156	450201	float64	1860	0.004114	450201	443680	0.981460	0
weatherVar155	450201	float64	1860	0.004114	450201	437064	0.966825	2.69654
weatherVar154	450201	float64	1860	0.004114	450201	441270	0.976129	0.246032
weatherVar153	450201	float64	1860	0.004114	450201	436639	0.965885	1.17469
weatherVar152	450201	float64	1860	0.004114	450201	450166	0.995808	0
weatherVar123	450201	float64	1860	0.004114	450201	437178	0.967077	0.967749
weatherVar86	450201	float64	1860	0.004114	450201	438563	0.970141	0.126252
weatherVar121	450201	float64	1860	0.004114	450201	436885	0.966429	1.80419
weatherVar45	450201	float64	1860	0.004114	450201	443291	0.980600	0.81792
weatherVar32	450201	float64	1860	0.004114	450201	437175	0.967071	1.10424
weatherVar33	450201	float64	1860	0.004114	450201	436919	0.966505	0.852247
weatherVar34	450201	float64	1860	0.004114	450201	436930	0.966529	0.347657
weatherVar35	450201	float64	1860	0.004114	450201	433597	0.959156	0.811226
weatherVar36	450201	float64	1860	0.004114	450201	437272	0.967285	0.679922
weatherVar37	450201	float64	1860	0.004114	450201	436806	0.966255	0.00248082
weatherVar38	450201	float64	1860	0.004114	450201	434640	0.961463	0.169487
weatherVar39	450201	float64	1860	0.004114	450201	448907	0.993023	0
weatherVar40	450201	float64	1860	0.004114	450201	437135	0.966982	1.09083
weatherVar42	450201	float64	1860	0.004114	450201	432776	0.957340	0.253126
weatherVar44	450201	float64	1860	0.004114	450201	437166	0.967051	1.13594
weatherVar46	450201	float64	1860	0.004114	450201	449910	0.995242	0
weatherVar61	450201	float64	1860	0.004114	450201	436343	0.965230	0.0710921
weatherVar47	450201	float64	1860	0.004114	450201	436911	0.966487	1.07313
weatherVar48	450201	float64	1860	0.004114	450201	434830	0.961883	0.178608
weatherVar49	450201	float64	1860	0.004114	450201	437080	0.966861	2.21262
weatherVar50	450201	float64	1860	0.004114	450201	445537	0.985568	0
weatherVar51	450201	float64	1860	0.004114	450201	437177	0.967075	1.06136
weatherVar52	450201	float64	1860	0.004114	450201	437077	0.966854	1.13797
weatherVar53	450201	float64	1860	0.004114	450201	437134	0.966980	0.905694
weatherVar55	450201	float64	1860	0.004114	450201	436972	0.966622	0.253435
weatherVar57	450201	float64	1860	0.004114	450201	437116	0.966940	0.407403
weatherVar120	450201	float64	1860	0.004114	450201	436826	0.966299	0
weatherVar59	450201	float64	1860	0.004114	450201	442830	0.979580	0.614592
weatherVar31	450201	float64	1860	0.004114	450201	434210	0.960512	0.009624
weatherVar30	450201	float64	1860	0.004114	450201	438639	0.970309	1.00439
weatherVar29	450201	float64	1860	0.004114	450201	439408	0.972010	1.22822
weatherVar28	450201	float64	1860	0.004114	450201	436944	0.966560	0.144197
weatherVar1	450201	float64	1860	0.004114	450201	432816	0.957428	0.182784
weatherVar2	450201	float64	1860	0.004114	450201	447856	0.990698	0
weatherVar3	450201	float64	1860	0.004114	450201	437148	0.967011	1.12618
weatherVar5	450201	float64	1860	0.004114	450201	432726	0.957229	0.291717
weatherVar7	450201	float64	1860	0.004114	450201	437188	0.967100	1.10374
weatherVar8	450201	float64	1860	0.004114	450201	438917	0.970924	1.44452
weatherVar9	450201	float64	1860	0.004114	450201	446184	0.987000	0
weatherVar10	450201	float64	1860	0.004114	450201	436651	0.965912	1.01689
weatherVar11	450201	float64	1860	0.004114	450201	432843	0.957488	0.147343
weatherVar12	450201	float64	1860	0.004114	450201	437081	0.966863	1.96786
weatherVar13	450201	float64	1860	0.004114	450201	436207	0.964930	0.0223433
weatherVar14	450201	float64	1860	0.004114	450201	437221	0.967173	1.11152
weatherVar15	450201	float64	1860	0.004114	450201	437116	0.966940	1.15092
weatherVar16	450201	float64	1860	0.004114	450201	437161	0.967040	1.01194
weatherVar18	450201	float64	1860	0.004114	450201	436948	0.966569	0.282293
weatherVar20	450201	float64	1860	0.004114	450201	437257	0.967252	0.381844
weatherVar21	450201	float64	1860	0.004114	450201	436628	0.965861	0.674719
weatherVar22	450201	float64	1860	0.004114	450201	434863	0.961956	0.125078
weatherVar23	450201	float64	1860	0.004114	450201	436999	0.966681	0.148698
weatherVar24	450201	float64	1860	0.004114	450201	433081	0.958015	0.161928
weatherVar25	450201	float64	1860	0.004114	450201	436764	0.966162	0.00232253
weatherVar26	450201	float64	1860	0.004114	450201	432481	0.956687	0.503055
weatherVar27	450201	float64	1860	0.004114	450201	437024	0.966737	1.0112
weatherVar60	450201	float64	1860	0.004114	450201	436982	0.966644	0.152828
weatherVar58	450201	float64	1860	0.004114	450201	436682	0.965980	0.611287
weatherVar62	450201	float64	1860	0.004114	450201	436713	0.966049	0.00279663
weatherVar93	450201	float64	1860	0.004114	450201	437264	0.967268	0.771335
weatherVar63	450201	float64	1860	0.004114	450201	436473	0.965518	0.735235
weatherVar96	450201	float64	1860	0.004114	450201	435726	0.963865	0.75424
weatherVar97	450201	float64	1860	0.004114	450201	438298	0.969555	0.136047
weatherVar98	450201	float64	1860	0.004114	450201	437133	0.966978	0
weatherVar99	450201	float64	1860	0.004114	450201	438954	0.971006	0
weatherVar100	450201	float64	1860	0.004114	450201	437035	0.966761	1.01238
weatherVar101	450201	float64	1860	0.004114	450201	435743	0.963903	0.785233
weatherVar102	450201	float64	1860	0.004114	450201	449641	0.994647	0
weatherVar103	450201	float64	1860	0.004114	450201	449390	0.994092	0
weatherVar104	450201	float64	1860	0.004114	450201	436664	0.965940	0.00850162
weatherVar105	450201	float64	1860	0.004114	450201	437064	0.966825	1.50307
weatherVar106	450201	float64	1860	0.004114	450201	436956	0.966586	0.636448
weatherVar107	450201	float64	1860	0.004114	450201	436719	0.966062	0.468449
weatherVar108	450201	float64	1860	0.004114	450201	433627	0.959222	1.51788
weatherVar109	450201	float64	1860	0.004114	450201	437153	0.967022	0.553574
weatherVar110	450201	float64	1860	0.004114	450201	437045	0.966783	0.000595711
weatherVar111	450201	float64	1860	0.004114	450201	436611	0.965823	0
weatherVar112	450201	float64	1860	0.004114	450201	437243	0.967221	0.959892
weatherVar114	450201	float64	1860	0.004114	450201	432778	0.957344	0.0156226
weatherVar116	450201	float64	1860	0.004114	450201	437269	0.967279	0.945053
weatherVar117	450201	float64	1860	0.004114	450201	446339	0.987342	0
weatherVar118	450201	float64	1860	0.004114	450201	449133	0.993523	0
weatherVar119	450201	float64	1860	0.004114	450201	436740	0.966109	0.98645
weatherVar94	450201	float64	1860	0.004114	450201	436882	0.966423	0.789749
weatherVar95	450201	float64	1860	0.004114	450201	440881	0.975269	0
weatherVar91	450201	float64	1860	0.004114	450201	436863	0.966381	0.72342
weatherVar75	450201	float64	1860	0.004114	450201	435969	0.964403	0.153111
weatherVar64	450201	float64	1860	0.004114	450201	437021	0.966730	1.01221
weatherVar65	450201	float64	1860	0.004114	450201	436957	0.966589	0.147134
weatherVar66	450201	float64	1860	0.004114	450201	447594	0.990119	5.58698
weatherVar67	450201	float64	1860	0.004114	450201	447306	0.989482	4.6836
weatherVar68	450201	float64	1860	0.004114	450201	436352	0.965250	0.0589026
weatherVar69	450201	float64	1860	0.004114	450201	437202	0.967131	1.09378
weatherVar70	450201	float64	1860	0.004114	450201	437002	0.966688	0.949523
weatherVar71	450201	float64	1860	0.004114	450201	436690	0.965998	0.348595
weatherVar89	450201	float64	1860	0.004114	450201	437142	0.966998	0.892934
weatherVar73	450201	float64	1860	0.004114	450201	437061	0.966819	0.665584
weatherVar74	450201	float64	1860	0.004114	450201	436776	0.966188	0.00278463
weatherVar72	450201	float64	1860	0.004114	450201	433987	0.960019	0.934886
weatherVar76	450201	float64	1860	0.004114	450201	437055	0.966805	1.57925
weatherVar84	450201	float64	1860	0.004114	450201	436527	0.965637	0.16871
weatherVar88	450201	float64	1860	0.004114	450201	436966	0.966608	1.67764
weatherVar78	450201	float64	1860	0.004114	450201	432659	0.957081	0.724788
weatherVar85	450201	float64	1860	0.004114	450201	436922	0.966511	1.59227
weatherVar87	450201	float64	1860	0.004114	450201	437084	0.966870	1.5284
weatherVar83	450201	float64	1860	0.004114	450201	436638	0.965883	0.926208
weatherVar82	450201	float64	1860	0.004114	450201	449948	0.995326	0
weatherVar81	450201	float64	1860	0.004114	450201	444965	0.984303	3.87028
weatherVar80	450201	float64	1860	0.004114	450201	436950	0.966573	1.52426
var7	451205	float64	856	0.001894	451205	451197	0.998089	3
target	452061	float64	0	0.000000	452061	450873	0.997372	0
dummy	452061	object	0	0.000000	452061	452060	0.999998	A
var17	452061	float64	0	0.000000	452061	427464	0.945589	84.3373
var13	452061	float64	0	0.000000	452061	451729	0.999266	0
var11	452061	float64	0	0.000000	452061	415222	0.918509	705.907
var10	452061	float64	0	0.000000	452061	451956	0.999768	3.89895
id	452061	int64	0	0.000000	452061	1	0.000002	1

Capstone Project - Liberty Mutual Group Fire Peril Loss Cost¶

Table of Contents¶

1. Objectives & Intention ¶

2. Importing the Data ¶

3. Cleaning the Data ¶

4. Exploratory Data Analysis ¶

5. Feature Engineering & Encoding ¶

5.1. Censoring Large Claims and Log-Transformation¶

5.2. Feature Compression: Principal Component Analysis¶

5.3. Downsampling of majority events¶

5.4. Perform Preprocessing¶

6. Model Evaluation Metric: Normalised Weighted Gini Coefficient¶

7. Model Building ¶

7.1. Model Building Process ¶

7.2. Regression & Classification Ensemble ¶

7.3. Ensembling: Ensemble Stacking & Blending ¶

7.4. Ensembling: Training a Meta Learner¶

7.4. Ensembling: Validating the Meta Learner ¶

8. Final Predictions & Submission ¶

9. Conclusion & Discussion ¶

9.1 Conclusion¶

9.2 Discussion¶

10. References ¶

6. Model Evaluation Metric: Normalised Weighted Gini Coefficient ¶

7.4. Ensembling: Training a Meta Learner ¶